Forum Discussion
Incremental refresh Rest Api best practice
Hi Anonymous
You can try setting up incremental refresh on a table from a REST API data source, but it may not be able to improve the refresh speed as you expected. I extract related paragraph from Incremental refresh for datasets in Power BI - Power BI as below.
=======================================
Incremental refresh works best for structured, relational data sources, like SQL Database and Azure Synapse, but can also work for other data sources. In any case, your data source must support the following:
Date column - The table must contain a date column of date/time or integer data type. The RangeStart and RangeEnd parameters (which must be date/time data type) filter table data based on the date column. For date columns of integer surrogate keys in the form of yyyymmdd, you can create a function that converts the date/time value in the parameters to match the integer surrogate key of the data source table. To learn more, see Configure incremental refresh - Convert DateTime to integer.
Query folding - Incremental refresh is designed for data sources that support query folding, which is Power Query's ability to generate a single query expression to retrieve and transform source data. Most data sources that support SQL queries support query folding. Data sources like flat files, blobs, and some web feeds often do not.
When incremental refresh is configured, a Power Query expression that includes a date/time filter based on the RangeStart and RangeEnd parameters is executed against the data source. The filter is in effect a transformation included in the query that defines a WHERE clause based on the parameters. In cases where the filter is not supported by the data source, it cannot be included in the query expression. The query mashup engine compensates and applies the filter locally, which requires retrieving all rows for the table from the data source. This can cause incremental refresh to be slow, and the process can run out of resources either in the Power BI service or in an On-premises Data Gateway - effectively defeating the purpose of incremental refresh.
Before configuring your incremental refresh solution, be sure to thoroughly read and understand Query folding guidance in Power BI Desktop and Power Query query folding. These articles can help you determine if your data source and queries support query folding.
=======================================
Per my understanding, REST API data sources probably don't support query folding, so setting up incremental refresh for it can help reduce the amount of data but perhaps cannot improve refresh speed.
Best Regards,
Community Support Team _ Jing
If this post helps, please Accept it as Solution to help other members find it.