This time we’re going bigger than ever. Fabric, Power BI, SQL, AI and more. We're covering it all. You won't want to miss it.
Learn moreDid you hear? There's a new SQL AI Developer certification (DP-800). Start preparing now and be one of the first to get certified. Register now
Hi,
I have a power bi query where I can retrieve items up to the page size limit of 500 but want all the items.
I have the following working code:
let
// Get Authorization Token
BearerToken = (Json.Document(Web.Contents("https://id.sophos.com/api/v2/oauth2/token",
[
Headers = [#"Content-Type"="application/x-www-form-urlencoded"],
Content = Text.ToBinary("grant_type=client_credentials&client_id=" & #"SophosClientID" & "&client_secret=" & #"SophosClientSecret" & "&scope=token")
]
)) [access_token]),
url=url="https://api-us03.central.sophos.com/",
PartnerIDQuery = Json.Document(Web.Contents(url, [RelativePath="endpoint/v1/endpoints?pageSize=500&pageTotal=true", Headers = [#"X-Tenant-ID"=tenantID,#"Authorization"="Bearer " & BearerToken]])),
items = PartnerIDQuery[items],
#"Converted to Table" = Table.FromList(items, Splitter.SplitByNothing(), null, null, ExtraValues.Error)
in
items
From the API documention:
The <from-key> example value in the pages field in the response above is the key associated with the first item on the page.
The <next-key> example value in the pages field in the response is the key you should use to fetch the next page.
As an example, the second page of endpoints for the first tenant could be fetched as follows:
curl -XGET -H "Authorization: Bearer <jwt>" \
-H "X-Tenant-ID: 4F426605-CE10-41A7-8BC7-24B5E8F4C4BF" \
https://api-us01.central.sophos.com/endpoint/v1/endpoints?pageFromKey=<next-key>The response from this API shows the value to use for pageFromKey to fetch the third page (if there is one). If the nextKey value is missing, you have reached the end of pages and can stop iterating.
I can obtain the Next key :
pages = PartnerIDQuery[pages],
nextKey1 = pages[nextKey]
How can i build a loop to iterate through all of the pages retrieving the items for each key?
Hi @ShaunBuchan ,
Please let us know if what @mahoneypat provided could solve your problem.
If it could, please accept his reply as solution to your question so that people who may have the same question can get the solution directly.
If not, please let me know.
In additon, here is one post may helo you: Iterate over dynamic Web API Pages with Power Query - How to resolve Cursor-Based Pagination - DataC....
Best Regards,
Icey
If this post helps, then consider Accepting it as the solution to help other members find it faster.
If the API has skip or offset functionality, please see if this video can show an alternate/simpler approach.
Power BI - Tales From The Front - REST APIs - YouTube
Pat
To learn more about Power BI, follow me on Twitter or subscribe on YouTube.
Check out the April 2026 Power BI update to learn about new features.
Sign up to receive a private message when registration opens and key events begin.
If you have recently started exploring Fabric, we'd love to hear how it's going. Your feedback can help with product improvements.