Forum Discussion
Solved
Automatic pagination with incrementing offset
Hi, I have a problem with my data source. I am using an external API which only allows me to show 100-200 projects at once. Unfortunately it will not show me the total pages. I hoped that this thre...
Please review the following blog to implement REST API pagination in Power Query.
https://medium.com/@marktiedemann/how-to-do-pagination-in-power-query-430460c17c78Paul Zheng _ Community Support Team
If this post helps, please Accept it as the solution to help the other members find it more quickly.
Thex You can use a Do Until loop in Power Automate for pagination if you can create a connection to your data source in Power Automate.
- Use 'Initialise Variable', name it Offset, select Integer and set the initial value to 0
- Create a Do Until loop with the Offset created above equal to the largest number possible (since you do not know the total pages
- Inside the Do Until loop, use ‘List Records’ to list the records you want from your data source. Here, use limit as 100 and offset as the Offset variable created above.
- You can add other actions within the Do Until loop such as storing these projects to your desired location for example, a SQL server or a database
- For the last step within the Do Until loop, use ‘Increment Variable’. Select Offset and the increment value as 100
- Run the flow
This loop will first get 100 records, then the next 100 records and so on until the largest number you have passed is met.
Hope this helps anybody facing a similar issue.