Forum Discussion
how to fully load data from limited HTTP API?
please, I still need help.
I tried to modify the script from another post, but then Power BI freezes for a second and then returns and error.
error: An error occured when deserializing the results. try to repeat the operation.
below is the full script that I would like someone to review and correct.
in bold, commands that I don´t understand, but I am using anyway because I just adapted from another post, and I think this is the way to get my solution.
let
Source = Json.Document(Web.Contents("https://url.com/token?limit=100&offset=0&date_range=201001010000:201912310000")),
Pagination = List.Skip(List.Generate( () => [offsett = "0"], // Start Value
each Source[offset] <> Source[next_offset], // Condition under which the next execution will happen
each [ WebCall = "https://url.com/token?limit=100&offset="&[offsett]&"date_range=201001010000:201912310000", // retrieve results per call
offsett = if WebCall[next_offset] >= WebCall[offset] then WebCall[next_offset] else WebCall[next_offset] ],// determine the offset for the next execution
each [Value]),1) // Select just the Record of the last step from your query
in
Pagination
I tried to wrap the webcall under Json.Document(Web.Contents()), but this also doesn´t work.
for information, the first line (Source) returns this, which is ok.
i need to use and compare the values from offset and next_offset.
the from_cache value is not good for this script, because it returns false or true even if I still have more data to read.
I tried to improve the last code as it follows below.
in bold, thing that i changed.
now i got a different erro: The type of the current preview value is too complex to display
but before the error occurs, i can see at the bottom right corner of the power bi, an increasing number of rows that goes a bit above 300 rows and then it crashes.
let
Source = Json.Document(Web.Contents("https://url.com/token?limit=100&offset=0&date_range=201001010000:201912310000")),
Pagination = List.Skip(List.Generate( () => [offsett = "0"], // Start Value
each Source[offset] <= Source[next_offset], // Condition under which the next execution will happen
each [ WebCall = "https://url.com/token?limit=100&offset=" & [offsett] & "&date_range=201001010000:201912310000", // retrieve results per call
offsett = WebCall[next_offset] ],// determine the offset for the next execution
each [posts]),1) // Select just the Record of the last step from your query
in
Pagination