Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

View all the Fabric Data Days sessions on demand. View schedule

Allow incremental refresh to create a partition for future data in import mode

Incremental Refresh allows you to use a DirectQuery partition for future data. It would be helpful to be able to configure that for use with import mode, too.


I have a large fact table that includes years of past data and many months of future data. DirectQuery is not performant enough for my users and I'd like to import the future data into a single partition. Then, each time it's refreshed it will refresh the recent past partitions identified in the configuration plus the future partition.

Status: New
Comments
Nick_VanLent
New Member

I completely agree. The data source we're using for some of our dataflows does not have a "record created on" field, so we're using a fiscal period start date field for incremental refresh instead. The problem is that the data contains forecast information for future periods, so incremental refresh doesn't pick this up. The workaround we're using is to have one query for current and past periods, another query for future periods, and a third query to append the first two. We setup incremental refresh only on the current/past periods query. The overall refresh time is a little faster, but not by much, because the append query takes additional time to run. Another downside is that we have more than one query set to "enable load", and the users can get confused when selecting the dataflow to pull into their datasets.

fbcideas_migusr
New Member
Status changed to: New