Showing results for 
Search instead for 
Did you mean: 

Fabric is Generally Available. Browse Fabric Presentations. Work towards your Fabric certification with the Cloud Skills Challenge.

Resolver II
Resolver II

How to overcome API call row limit?



I am attempting to extract measurement data from an API with a limit of 1000 rows per call. Each row consists of a data value and a timestamp. I have set up a Power Query script that connects to the API and can retrieve any 1000 concecutive rows of data. 


What I need is an iterative script that retrieves the data I need in chunks of 1000 rows and merges them ino a single table. Below is my current script. The Source line is the one of interest here.




    Source = Json.Document(Web.Contents(""&url&"/TagsData/"&tagid&"/After?subscription-key="&key&"&after="&after&"&count="&count&"")),
    #"Converted to Table" = Table.FromList(Source, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"tag", "data"}, {"tag", "data"}),
    #"Removed Columns" = Table.RemoveColumns(#"Expanded Column1",{"tag"}),
    #"Expanded data" = Table.ExpandListColumn(#"Removed Columns", "data"),
    #"Expanded data1" = Table.ExpandRecordColumn(#"Expanded data", "data", {"d", "v"}, {"d", "v"}),
    #"Changed Type" = Table.TransformColumnTypes(#"Expanded data1",{{"d", type datetime}, {"v", type number}})
    #"Changed Type"




 The body parameter "after" sets the datetime after which you want data from. The parameter "count" sets the number of rows, and is limited to 1000.


One challenge is that the API is not paginated, so I will need to keep track of timestamps and number of returned rows for each call.


I have been working with PowerBI professionally for the last year, but my Power Query scripting skills are not good enough for this challenge. Thanks in advance for any help.


Cheers, Mike

Frequent Visitor

@mahoneypat Hey Pat, can't you use the theories and code in this community post to update the Salesforce Report connector to automatically call the next set of rows until it returns blank?


Instead of pagination, you can also use List.Numbers along with Skip or Offset in the web calls to overcome the row limit.  This video walks through it.

Power BI - Tales From The Front - REST APIs - YouTube




Did I answer your question? Mark my post as a solution! Kudos are also appreciated!

To learn more about Power BI, follow me on Twitter or subscribe on YouTube.

@mahoneypa HoosierBI on YouTube

@mahoneypat Thanks for that wonderful video. I have implemented skip and offset method while pulling data from API. Earlier it used to limit 100 records but using this I am able to load complete data. Next challenge is refresh doesn't work in Power BI Service. Returns error saying This dataset contains dynamic data source. 

Any suggestions so that refresh works..Please note I already tried approach mentioned in below link

but then after adding skip and top as query parameters.. it pulls same set of records everytime.. so this is not working for me. Do you suggest any other option which will resolve dynamic data source issue in Service?

Community Champion
Community Champion

Hello @h4tt3n 


there are a lot of examples of how to do in this forum.

They use all List.Generate where you have a parameter when to stop to generate to api calls like giving an error or empty page.

just start out to search for api List.Generate.

If you have any specific question feel free to come back

If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too

Have fun


Helpful resources

PBI November 2023 Update Carousel

Power BI Monthly Update - November 2023

Check out the November 2023 Power BI update to learn about new features.

Community News

Fabric Community News unified experience

Read the latest Fabric Community announcements, including updates on Power BI, Synapse, Data Factory and Data Activator.

Dashboard in a day with date

Exclusive opportunity for Women!

Join us for a free, hands-on Microsoft workshop led by women trainers for women where you will learn how to build a Dashboard in a Day!

Power BI Fabric Summit Carousel

The largest Power BI and Fabric virtual conference

130+ sessions, 130+ speakers, Product managers, MVPs, and experts. All about Power BI and Fabric. Attend online or watch the recordings.

Top Solution Authors