The ultimate Fabric, Power BI, SQL, and AI community-led learning event. Save €200 with code FABCOMM.
Get registeredEnhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.
We have Dataflows that utilize API Sources that we would like to incrementally refresh. These are working as expected since the source filters the data to the latest snapshot data which has a Data Through Date that allows us to detect the period we want to refresh. We are using incremental to support reporting on current and prior period data.
The issue we are seeing is that each time the incremental refresh runs, Power BI makes multiple calls to the source API. While this doesn't present a problem for most of our sources, we have a couple that provide larger result sets and are memory intensive. Making multiple calls to the source (5-8x within 30 seconds of the refresh starting) uses too much memory and ultimately brings down that API.
When we run full refreshes, the Power BI Service only calls the source once.
A couple of things to note:
Any idea why the Source API would be called so many times in rapid succession?
When you say Incremental Refresh are you referring to the Power BI version (including RangeStart and RangeEnd datetime filters for the data source, and Power BI's partition management), or something that you have created manually in your query?
User | Count |
---|---|
35 | |
14 | |
12 | |
11 | |
8 |
User | Count |
---|---|
43 | |
43 | |
19 | |
19 | |
18 |