Forum Discussion
'Dynamic data source' error for Planned Refresh on semantic model
Thanks. Here's the API.
I didn't make the 10000 requests part myself. But it was necessary because the API only allowed 100 records at a time.
Is this a problem for a planned refresh?
let
BaseURL = "https://api.onview.nl/ETCETCETC",
PageSize = 100,
MaxLimit = 10000,
PagesToFetch = List.Generate(
()=> [PageNumber=1, Offset=0],
each [Offset] < MaxLimit,
each [PageNumber=[PageNumber]+1, Offset=[Offset]+PageSize]
),
AllData = List.Transform(PagesToFetch, (page) =>
let
Response = Json.Document(
Web.Contents(
BaseURL &
"?offset=" & Text.From(page[Offset]) &
"&limit=" & Text.From(PageSize),
[Headers=[#"Content-Type"="application/json"]]
)
)
in
Response
),
CombinedData = Table.FromList(AllData, Splitter.SplitByNothing()),
#"Expanded Column1" = Table.ExpandRecordColumn(CombinedData, "Column1", {"Result", "TotalItems", "NextPage", "Content"}, {"Column1.Result", "Column1.TotalItems", "Column1.NextPage", "Column1.Content"}),
#"Removed Result/Total/NextPage" = Table.RemoveColumns(#"Expanded Column1",{"Column1.Result", "Column1.TotalItems", "Column1.NextPage"}),
#"Expanded Column1.Content" = Table.ExpandListColumn(#"Removed Result/Total/NextPage", "Column1.Content"),
#"Expanded Column1.Content1" = Table.ExpandRecordColumn(#"Expanded Column1.Content", "Column1.Content", {"RecId", "DossierId", "WorkflowId", "WorkflowTitle"}, {"WorkflowRegelnr", "DossierRecId", "Workflowtype.RecId", "Workflow.Omschrijving"}),
#"Filtered DossierRecId <> Test" = Table.SelectRows(#"Expanded Column1.Content1", each ([DossierRecId] <> 374)),
#"LET OP: hierna negeer ik regelnr" = #"Filtered DossierRecId <> Test",
#"Removed Duplicates behalve regnr" = Table.Distinct(#"LET OP: hierna negeer ik regelnr", {"DossierRecId", "Workflowtype.RecId", "Workflow.Omschrijving"}),
#"Added Workflowsleutel" = Table.AddColumn(#"Removed Duplicates behalve regnr", "WorkflowSleutel", each ([DossierRecId] * 1000) + [Workflowtype.RecId]),
#"Merged CaseloadZwaarte" = Table.NestedJoin(#"Added Workflowsleutel", {"Workflowtype.RecId"}, CaseloadZwaarte, {"WF type nr"}, "CaseloadZwaarte", JoinKind.LeftOuter),
#"Expanded CaseloadZwaarte" = Table.ExpandTableColumn(#"Merged CaseloadZwaarte", "CaseloadZwaarte", {"Caseload punten"}, {"Caseload punten"}),
#"Changed Type" = Table.TransformColumnTypes(#"Expanded CaseloadZwaarte",{{"WorkflowRegelnr", Int64.Type}, {"DossierRecId", Int64.Type}, {"Workflowtype.RecId", Int64.Type}, {"Workflow.Omschrijving", type text}, {"WorkflowSleutel", Int64.Type}}),
#"Removed Duplicates" = Table.Distinct(#"Changed Type", {"WorkflowRegelnr"}),
#"Filtered eventuele fouten" = Table.SelectRows(#"Removed Duplicates", each [WorkflowRegelnr] <> null and [WorkflowRegelnr] <> ""),
#"Appended Query" = Table.Combine({#"Filtered eventuele fouten", #"Acties (Task) zonder workflow"})
in
#"Appended Query"Yes, that looks like the culprit. The first parameter to web.contents should be static, only "https://api.onview.nl". Then use the RelativePath and Query options in web.contents to build out the remainder of the URL. You can take a look at the following for RelativePath and Query options: https://blog.crossjoin.co.uk/2016/08/16/using-the-relativepath-and-query-options-with-web-contents-in-power-query-and-power-bi-m-code/
However, I have found working with REST APIs in PowerQuery is like a puzzle, as there may be other issues you run into (i.e. authentication) that may not be easily solved. What works in the Desktop does not always work in the Service.
Good luck.