Incremental Refresh

[kakarot13]

Hi Everyone,

I have imported my data using a SQL query (Oracle DB) into PBI Desktop. I've tried to implement Incremental refresh (followed the steps of creating RangeStart and RangeEnd parameters and applied custom filtering on the date/time column). I've enabled incremental refresh but got a warning that says "Unable to confirm if the M query can be folded. It is not recommended to use incremental refresh with non-foldable queries".

1. Can power bi service ignore the above warning and perform incremental refresh?

2. If Power bi service cannot perform incremental refresh, is there a way to manipulate the source query to enable query folding?

My dataset has more than 10 million rows and growing everyday. Please advise if there's any alternate approach.

Thanks!

[Icey · Accepted solution]

Hi @kakarot13 ,

---

1. Can power bi service ignore the above warning and perform incremental refresh?

---

If query folding is not supported with the queries, incremental refresh can be set up. However, it would be very slow and might not work as planned. The incremental load won’t make the refresh process that much different, because the data will be loaded fully from the source to the engine of Power Query every time, and filtered there.

---

2. If Power bi service cannot perform incremental refresh, is there a way to manipulate the source query to enable query folding?

---

Please determine when a query can be folded.

When you identify that certain Power Query query steps can't be folded, it may be possible to apply the transformations in the data source. The transformations could be achieved by writing a database view that logically transforms source data. Or, by physically preparing and materializing data, in advance of Power BI querying it. A relational data warehouse is an excellent example of prepared data, usually consisting of pre-integrated sources of organizational data.

Best Regards,

Icey

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.