Pagination with cursor rather than offset

[thisisausername]

I need to paginate through a dataset that has a 'next' value of the last DocumentId in each page of data.

How can I create a query that lets me do the following:

1) Loads initially a 1st URL that contains the root search query (i.e. has no filter on DocumentId)

e.g. www.test.com/powerbi/search

2) From this first page, get the last DocumentId. (This takes a few transformation steps)

3) Load the next page with a 2nd URL which contains a filter for DocumentId = the last DocumentId from the previous page. 

e.g. www.test.com/powerbi/search?$cursor=LastDocumentIdFromPreviousPage

4) Keep repeating steps 2) and 3) until there is no more data. 

I've tried to look into this but can't find an example which shows how to have step 2) occur using multiple transformation steps (not just a single step). Thank you.

[miguel]

Hey!

Here's an article that might help you:

Helper functions for M extensions for Power Query connectors - Power Query | Microsoft Docs

Best!