ServiceNow API - Pagination

[Ruairi_CloudP]

Hello all, 

We're currently building Power BI reporting for a client where we are using REST API endpoints to retreive data from their ServiceNow environment to report on. 

The APIs in question are custom APIs which have longer processing times, where we are bringing through ServiceNow rows which are of a certain 'Survey Type' into mutliple tables. One table that has been having issues is one for 'Support' surveys, as this has a larger number of rows to bring through and has longer processing times than the other tables, which leads to some timeout issues. We've had a consistent issue with this table bringing through different numbers of rows every time it is refreshed where it does not bring through every row which we need. 

We're attempting to paginate this so it processes in chunks of 1,000 rows to resolve this issue however we've been struggling to put this together. We're looking to paginate this where we're not calling on a specific numbers of pages but just spliting the processing up. 

Any help with this would be appreciated, thanks!

[amitchandak]

@Ruairi_CloudP , refer if these can help

https://medium.com/@marktiedemann/how-to-do-pagination-in-power-query-430460c17c78

https://datachant.com/2016/06/27/cursor-based-pagination-power-query/

[Ruairi_CloudP]

Hello, 

Thanks for the response, 

We've looked at the articles you've linked and tried using them as a basline to use for our pagination, however we haven't been able to get it to work yet. 

For the first link, we tried using this as a basis however we keep running into an error which is as the following: Web.Contents failed to get contents from "[URL Endpoint]" (406): Not Acceptable
Details:
DataSourceKind=Web
DataSourcePath= "[URL Endpoint]"

Url = "[URL Endpoint]"

For the second link, we have an issue where the result is returning a blank row of lists like so:

Any help with this? Thanks.

[ams1]

Hi @Ruairi_CloudP 

Do you still have this problem regarding connecting to service now?

If yes, maybe I can try to help out - I would at least need from you the link to the service now API endpoint you'd like to use (ideally I would also like to see the query - AFTER you remove confidential information from it).