Join us at FabCon Atlanta from March 16 - 20, 2026, for the ultimate Fabric, Power BI, AI and SQL community-led event. Save $200 with code FABCOMM.
Register now!The Power BI Data Visualization World Championships is back! Get ahead of the game and start preparing now! Learn more
Hi,
I have a power bi query where I can retrieve items up to the page size limit of 500 but want all the items.
I have the following working code:
let
// Get Authorization Token
BearerToken = (Json.Document(Web.Contents("https://id.sophos.com/api/v2/oauth2/token",
[
Headers = [#"Content-Type"="application/x-www-form-urlencoded"],
Content = Text.ToBinary("grant_type=client_credentials&client_id=" & #"SophosClientID" & "&client_secret=" & #"SophosClientSecret" & "&scope=token")
]
)) [access_token]),
url=url="https://api-us03.central.sophos.com/",
PartnerIDQuery = Json.Document(Web.Contents(url, [RelativePath="endpoint/v1/endpoints?pageSize=500&pageTotal=true", Headers = [#"X-Tenant-ID"=tenantID,#"Authorization"="Bearer " & BearerToken]])),
items = PartnerIDQuery[items],
#"Converted to Table" = Table.FromList(items, Splitter.SplitByNothing(), null, null, ExtraValues.Error)
in
items
From the API documention:
The <from-key> example value in the pages field in the response above is the key associated with the first item on the page.
The <next-key> example value in the pages field in the response is the key you should use to fetch the next page.
As an example, the second page of endpoints for the first tenant could be fetched as follows:
curl -XGET -H "Authorization: Bearer <jwt>" \
-H "X-Tenant-ID: 4F426605-CE10-41A7-8BC7-24B5E8F4C4BF" \
https://api-us01.central.sophos.com/endpoint/v1/endpoints?pageFromKey=<next-key>The response from this API shows the value to use for pageFromKey to fetch the third page (if there is one). If the nextKey value is missing, you have reached the end of pages and can stop iterating.
I can obtain the Next key :
pages = PartnerIDQuery[pages],
nextKey1 = pages[nextKey]
How can i build a loop to iterate through all of the pages retrieving the items for each key?
Hi @ShaunBuchan ,
Please let us know if what @mahoneypat provided could solve your problem.
If it could, please accept his reply as solution to your question so that people who may have the same question can get the solution directly.
If not, please let me know.
In additon, here is one post may helo you: Iterate over dynamic Web API Pages with Power Query - How to resolve Cursor-Based Pagination - DataC....
Best Regards,
Icey
If this post helps, then consider Accepting it as the solution to help other members find it faster.
If the API has skip or offset functionality, please see if this video can show an alternate/simpler approach.
Power BI - Tales From The Front - REST APIs - YouTube
Pat
To learn more about Power BI, follow me on Twitter or subscribe on YouTube.
The Power BI Data Visualization World Championships is back! Get ahead of the game and start preparing now!