Managing pagination for a Web API connector used in a dataflow gen2

Hi pmscorca ,

Thanks for the follow-up questions

In many scenarios, Dataflow Gen2 is the preferred choice for lightweight ingestion and transformation workflows, especially when direct integration with Lakehouses is needed. However, when consuming data from paginated REST APIs, the lack of native managed pagination becomes a significant limitation.

Currently, users must implement manual pagination using recursive Power Query M scripts, which can be complex, error-prone, and difficult to maintain  particularly for APIs using cursor based tokens, header-driven continuation, or dynamic termination conditions.

This appears to be a current limitation in Power BI. I would recommend submitting this scenario as a feature request on the official Power BI Ideas forum. The Product Team actively monitors suggestions there, and items with strong community support are often considered for future enhancements.

You can submit your idea here