Cursor Paginated data from Web API with List type outcome

[paulorojog]

Hi everyone!

I am calling a web api from an app provider (https://api.medilink.healthatom.com/api/v1/) to grab some data which is paginated and I am struggling with how to get the next page of data as it is cursor based. I've researched in previous post and videos and the difference it seems to be the type of records.

Here is all the documentation from the API. I think especially maybe the coursors have some relation to the problem and would help to have more context: https://api.medilink.healthatom.com/docs/#cursores

Through my investigation, by the moment I've created the following function "GetData" and brings effectivelly 50 records, which is the expected behavior because the data is paginated in that number:

GetData Function

(url as text) as record =>
let
    Source = Json.Document(Web.Contents(url, [Headers = [Authorization="Token tokencode"]]))
in
    Source

Result

Open data there are the 50 records, record has the link from the current and next page:

...

Then I've tried to add a pagination query based on other posts but I'm receiving the following error from the row 2 onwards; the first row has the first 50 records:

Query

let
myList = List.Generate( ()=>
    [Result = GetData("https://api.medilink.healthatom.com/api/v1/prestaciones"), Counter = 0],
    each  [Counter] < 5,
    each [Result =  GetData(Table.LastN([Result]{[Name = "link"]}[Table], 1){0}[#"Attribute:href"]), Counter = [Counter]+1],
    each [Result])
in
    myList

Result

Some related content I've already read is:

https://community.powerbi.com/t5/Desktop/Cursor-Paginated-data-from-Web-API/td-p/1029953/page/1

https://datachant.com/2016/06/27/cursor-based-pagination-power-query/

Thank you!

[v-alq-msft]

Hi, @paulorojog 

Are there any error messages in the query? You may try Record.Combine to bring records together.

Best Regards

Allan

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[paulorojog]

Hi everyone, 

I'm still working on this. I've created a second function that I think is working, but still I need some help. The problem now is I have different GeneratedList{0}, {1}... but I can not see it all. How could I bring together all lists?

Function GetData2 ("links" is the record where the coursors of the API are located)

(url as text) as record =>
   let
    Source = Json.Document(Web.Contents(url, [Headers = [Authorization="Token tokencode"]])),
    data = try Source[data] otherwise null,
    next = try Source[links][next] otherwise null,
    res = [Data=data, Next=next]
   in
    res

Query (10 iteraciones)

let
 GeneratedList =
  List.Generate(
   ()=>[i=0, res = GetData2("https://api.medilink.healthatom.com/api/v1/prestaciones")],
   each [i]<10 and [res][Data]<>null,
   each [i=[i]+1, res = GetData2([res][Next])],
   each [res][Data])
in
    GeneratedList

Result

List with 1 row "Error" (no info about the error). But if I open that error row, I got 50 records from the GerenatedList{0}. Then if I change 0 per 1 or two, I got the other records but they are all separated.

Please any help would be appreciated! Thank you!