Forum Discussion
Problem with GitLab API pagination in Power BI - Limited response size
Hello everyone,
I am facing a problem with the integration of the GitLab API in Power BI. I am using the GitLab API to retrieve issues from our company and I am running into a limitation with pagination. Although there are about 5400 issues in total in our GitLab:
I only ever get 258 issues back via the API queries in Power BI (after adjusting the per_page header. Otherwise there would only be 20 records in response due to the default GitLab API return limit).
Even when docking the endpoint via Postman with keyset pagination, I only get 258 for X-Total:
running this GET request: https://gitlab.company_name.com/api/v4/issues?pagination=keyset&per_page=2&order_by=updated_at&sort=asc
The Power-Query Script im currently using in the Dataflow:
But Response column is just 258 records as well:
I appreciate any help!
Best regards 🙂
Fixed it. Needed to set the header parameter "scope=all" as described in the api docs for gitlab: https://docs.gitlab.com/ee/api/issues.html
4 Replies
Hi Anonymous
Based on your description, you can refer to the following link.
curl - Pagination in Gitlab API only returns 100 per_page max - Stack Overflow
Solved: API limited to 50 responses - pagination looping r... - Microsoft Fabric Community
Best Regards!
Yolo Zhu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Thanks for your quick answer!
In my described approach to the power query script i already tried to call the API iteratively until i get back pages without content. but so far this only works so well that i get 258 records as return instead of 100 (because 100 is the max number of returns per page). And don't know at all why that is because the total issue number on the gitlab server is about 5k.
Fixed it. Needed to set the header parameter "scope=all" as described in the api docs for gitlab: https://docs.gitlab.com/ee/api/issues.html