Hello,
don;t know why this is happening and what and how we can fix this issue.
let
// M query script for extracting data from domo using export method
// calling the oauth endpoint for getting the bearer token from domo
AccessToken = Json.Document(Web.Contents("https://api.domo.com/oauth/token?grant_type=client_credentials", [Headers=[Authorization=BasicAuth]])),
// converting result body to table
#"Converted to Table" = Table.FromRecords({AccessToken}),
// changing type for access_token column to text
#"Changed Type Access_Token" = Table.TransformColumnTypes(#"Converted to Table",{{"access_token", type text}}),
// storing the first row value of access_token column into a variable
access_token = #"Changed Type Access_Token"{0}[access_token],
// calling domo export data method endpoint
Export = Csv.Document(Web.Contents("https://api.domo.com/v1/datasets/"&DatasetID&"/data?includeHeader=true&fileName=data-dump01.csv", [Timeout=#duration(0, 0, 0, 10), Headers=[Authorization="Bearer " & access_token]]),[Delimiter=",", Encoding=1252, QuoteStyle=QuoteStyle.Csv]),
// extracting header information from first row
#"Promoted Headers" = Table.PromoteHeaders(Export, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"AmountTransactionTotal", Currency.Type}, {"MasterDate", type date}, {"Quantity", type number}, {"Contribution%", Percentage.Type}, {"CustomerUpdateDate", type date}, {"TransactionDate", type date}, {"LeadCreatedDate", type date}, {"CreatedDate", type date}, {"LeadCreatedDateTime", type datetime}, {"CustomerUpdateDateTime", type datetime}, {"DateofLastStatusChange", type date}, {"LeadScore", Int64.Type}}),
#"Replaced Value" = Table.ReplaceValue(#"Changed Type","",null,Replacer.ReplaceValue,{"Subsidiary"}),
#"Replaced Value1" = Table.ReplaceValue(#"Renamed Columns","",null,Replacer.ReplaceValue,{"ID"}),
#"Replaced Value2" = Table.ReplaceValue(#"Replaced Value1","",null,Replacer.ReplaceValue,{"Object"}),
#"Renamed Columns1" = Table.RenameColumns(#"Replaced Value2",{{"TeamTerritory", "Team"}})
in
#"Renamed Columns1"
Issue is total dataset has around800k rows and it never gives us any issues when refreshed locally, but when set to refresh on schedule randomly some data will be missing if we check the rows it is around 80k or 100k or 200k. and in the powerbi service it shows refresh succeded.
not really understanding what we can do to fix the issue and what is happening in the first place so that i can point to the cause and do something, its a pretty wide dataset with over 200 columns and 800 - 900k rows.
