Forum Discussion
Datamart Incremental Refresh is Failing
I set up some tables in one of my datamarts to refresh incrementally with the below settings. After the schedule refresh, I can see today's and yesterday's data. But after some time, the previous day data just disappear and the tables only maintain the most recent data. For example, after today's scheduled refresh, the table contained data from October 16 and 17. After two hours, I noticed that the table has October 17 data only. I noticed this happening in two different days without any unscheduled refreshes, being triggerred, just randomly. Anyone knows what's wrong with my settings?
I also tried changing the refresh period for different values, and the datamart is spontantenously updated (deleting old data) even without a refresh.That is working as designed. Any meta data change will result in a reset of the incremental refresh partitions and a rebuild on the next refresh.
Incremental refresh is for import mode only. (Small exception when you have a "live" "today" partition that is working in Direct Query)
5 Replies
Do not use the same column for change detection. Use a "Modified Date" column, or don't do change detection at all.
Thanks for your reply. I tried disabling the "Refresh Changed Data" option, but it didn't solve my issue.
I also tried changing the refresh period for different values, and the datamart is spontantenously updated (deleting old data) even without a refresh. Could it be the storage mode? I can see there are three options for storage: direct query, import, and both. But I am not able to change it from "direct query"
I also tried changing the refresh period for different values, and the datamart is spontantenously updated (deleting old data) even without a refresh.That is working as designed. Any meta data change will result in a reset of the incremental refresh partitions and a rebuild on the next refresh.
Incremental refresh is for import mode only. (Small exception when you have a "live" "today" partition that is working in Direct Query)
