Forum Discussion
How to deal with paginated API
- 3 years ago
Hi kevinEGE ,
Find the total number of records available from the endpoint. It's usually the highest-level return from the endpoint and should look something like this:
Once you have that, then you can build a list of numbers split by your page limit (100 in your case), something like this:
Convert this list to a table, then add a new custom column, something like this:
Table.AddColumn( previousStepName, "newColumnName", each Json.Document( Web.Contents( "https://api.root.com/", [ RelativePath="relative/path.api", Query= [ api_key=apiKey, _start=Text.From([youNumberListColumnName]), _limit="100" // if required ] ] ) ) )Once you have your column filled with nested records you can expand it and start drilling down to your actual data:
Pete
Hi kevinEGE ,
Find the total number of records available from the endpoint. It's usually the highest-level return from the endpoint and should look something like this:
Once you have that, then you can build a list of numbers split by your page limit (100 in your case), something like this:
Convert this list to a table, then add a new custom column, something like this:
Table.AddColumn(
previousStepName,
"newColumnName",
each Json.Document(
Web.Contents(
"https://api.root.com/",
[
RelativePath="relative/path.api",
Query=
[
api_key=apiKey,
_start=Text.From([youNumberListColumnName]),
_limit="100" // if required
]
]
)
)
)
Once you have your column filled with nested records you can expand it and start drilling down to your actual data:
Pete