Forum Discussion
gotmike
10 years agoFrequent Visitor
how to create a query that paginates?
I'm working with the Hubspot CRM API and when you query for a list of all deals, you only get 100 records at a time, and they want you to send subsequent queries with an "offset" to paginate the resu...
shad0wca7
8 years agoFrequent Visitor
I spent a while on this recently and finally cracked it. Here's some code that will retrieve all companies from Hubspot - some entries may be duplicated (I never figured that out) but a simple next query step of 'remove duplicates' does the trick. It can be used as the basis for all Hubspot API calls and probably similar pagination APIs:
let
Pagination = List.Skip(List.Generate( () => [IsMore = null, Last_Key = 0, Counter = 0], // Start Value
each [IsMore] <> false,// Whilst this is true, keep going
each [WebCall = Json.Document(Web.Contents("https://api.hubapi.com/companies/v2/companies/paged?hapikey=" & #"Hubspot API Key" & "&properties=name&properties=website&limit=250&offset=" & Text.From([Last_Key]) & "")), // retrieve results per call
Last_Key = try [WebCall][offset] otherwise 0,
IsMore = if [Counter] < 1 then null else [WebCall][#"has-more"],
Counter = [Counter]+1,
Table = Table.FromRecords(WebCall[companies])
]
,each [Table] // selector
) ,1)
// in
// Pagination
,
Custom1 = Table.Combine(Pagination)
in
Custom1JackSelman
Helper I
3 years agoThanks for sharing shad0wca7 ! I don't suppose you've updated this with the changes to HubSpot APIs? I'm using a PAT and I'm struggling to figure out hot to get it to work (as opposed to the APIKEY