Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
4 years ago

Dynamic API Pagination with next page Cursor in a Record

Hello,

 

I am having issues with writing a custom Query to get some data from an API.


In the initial response, I get these 2 results:

  data - (that has the 1st page of the data needed) and "pagination" which has this:

 

In order to get the next page of data, we need to make another API request with the value from endCursor added to the API uri in this format (...api.com/blabla?after=endCursorValue)

 

I've looked trough everything on the forum here and on the internet but I still can't find a suitable solution for this issue.

 

I'm sure that I need some kind of a loop that will add all the data in one list and request the next page of data using the endCursor value till there are no next pages (endCursor = "";  or hasNextPage = FALSE)

 

Thanks for your support!

Need Help
Tutorial Requests

4 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable
    4 years ago

    Here is a copy of the Pagination code from the API documentation:

     

    import requests

url = "https://api.samsara.com/fleet/vehicles"
params = {}
headers = {
    "Authorization": "Bearer TOKEN"
}

hasNextPage = True
while hasNextPage:
    response = requests.request("GET", url, headers=headers, params=params).json()
    for vehicle in response["data"]:
        # process each vehicle
    hasNextPage = response["pagination"]["hasNextPage"]
    params["after"] = response["pagination"]["endCursor"]

     

    I'm having a hard time translating this into M script. Can someone help me out with the M code that would work?

  • AmandaMulryan's avatar
    AmandaMulryan
    Frequent Visitor
    4 years ago

    Did you ever figure this out? I am trying to do the same thing with Samsara and am so close but can't seem to get it exactly right. Any help would be greatly appreciated! Thanks

     

    Anonymous

  • AmandaMulryan's avatar
    AmandaMulryan
    Frequent Visitor
    4 years ago

    v-easonf-msft Anonymous

     

    Here is what my data looks like in regard to the API loop:

     

     

     

    Here is the query I've made but it does not work. It keeps saying "The field 'pagination.hasNextPage' of the record wasn't found. 

     

    I am in need of code assistance if anyone is willing to help me.