Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
xli629b
Helper II
Helper II

Incremental refresh support for a non-folding data source

A warning message in Power BI Desktop says "It is not recommended to use incremental refresh with non-foldable queries."


However, incremental refresh for a non-folding data source actually does seem to work. I'm wondering what the actual limitations are then for non-folding data sources?

For example, if I have a data source that accepts start/end dates in the query string, and I simply directly substitute in those start/end parameters into the string:

 

data = MyDataSource("[...], dates=range(" & RangeStart & "," & RangeEnd & ")")

 

I can get it to only query for the past month upon subsequent refreshes after querying for six months of historical data the first time.

Even if I add some kind of simple transformation afterwards such as SelectRows, it still doesn't seem to break incremental refresh.

Cases like this seem to suit our needs just fine.

The documentation warns:
If it’s a pure import mode policy without real-time data, the query mashup engine might compensate and apply the filter locally, which requires retrieving all rows for the table from the data source. 

But would this pitfall only apply if I try to apply SelectRows(..., each [start_data] >= RangeStart and [end_date] < RangeEnd) to do the date range filtering instead of the direct string substitution above? Or are there other things that can go wrong with non-folding data sources with incremental refresh?
1 ACCEPTED SOLUTION
ppm1
Solution Sage
Solution Sage

What you are doing is fine. By putting RangeStart and RangeEnd in the query, you are manually "folding".

 

Pat

Microsoft Employee

View solution in original post

1 REPLY 1
ppm1
Solution Sage
Solution Sage

What you are doing is fine. By putting RangeStart and RangeEnd in the query, you are manually "folding".

 

Pat

Microsoft Employee

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

Power BI Carousel June 2024

Power BI Monthly Update - June 2024

Check out the June 2024 Power BI update to learn about new features.

PBI_Carousel_NL_June

Fabric Community Update - June 2024

Get the latest Fabric updates from Build 2024, key Skills Challenge voucher deadlines, top blogs, forum posts, and product ideas.