Looping API (total results not available)

[Anonymous]

Hi,

Been asking a similar question before, but for a different API.

The current API gives me the ID of the last row much like this example from hubspot API

https://api.hubapi.com/crm/v3/objects/contacts?limit=10&archived=false&hapikey=demo

For the demonstrations sake i have set the limit to 10 row just to visualize how an "incomplete" call will look like

As you can see the two last columns is for the paging. It gives the last row ID in the "after" column and the full link of the next 10 rows in the "link" column.

If i set the limit to 100, all 33 rows in the Demo dataset is returned. As you can see, the "after" and "link" columns are not returned.

As mentioned, I have solved a similar issue before, but that API gave me the total number of rows available in the paging, enabling use of the List.Generate() way to do it by creating a loop table of [total number of results available] / [limit in each call] = number of calls.

How do I proceed with this when i don't know how many times to loop the call?

Thanks in advance

Wes

[mahoneypat]

Please see this video for another way to approach this.  If you API support offset or skip in your web call, you can leverage that after value and dynamically generate a list of numbers with the right spacing for the right number of calls.

Power BI - Tales From The Front - REST APIs - YouTube

Pat

[Anonymous]

Thanks @mahoneypat , your video was perfect for the previous API I was working with where I got the total number of rows available.

In the current API the pagination part of the call return only gives me the following.

after: the ID of the last result in the current call

link: full link of the next call, skipping all rows in the current call (&after=32401)

Also worth mentioning, the ID is not sequential. The next ID might be 33551, i.e. its not 32402.

Any hints?

Wes

[mahoneypat]

If you know approximately how many to expect, you could make a list of numbers to use as your "after" values that intentionally goes just past it and then remove any rows where you get error instead of "Table".  That would be simpler than a recursive solution.

Pat

[KNP]

Nice video @mahoneypat.

That's an elegant way to handle API calls.