Forum Discussion
Incremental Refresh isn't dropping old data
It does seem like it should be dropping the older data. Maybe it has to do with the way it partitions the data? If the data you want to drop is in the same partition as some of the data you still want, maybe it just keeps the whole partition.
This text from the documentation I linked previously makes me think it's preserving more than strictly the period you specify:
The Archive data starting before refresh date setting determines the historical period in which rows with a date/time in that period are included in the model, plus rows for the current incomplete historical period, plus rows in the refresh period up to the current date and time.
For example, if you specify five years, the table stores the last five whole years of historical data in year partitions. The table will also include rows for the current year in quarter, month, or day partitions, up to and including the refresh period.
I opened a support ticket with Microsoft on this. I have 2 Semantic Models that I know this has occurred on and then I have 2 models that are actual incremental models that are too big for me to delete and then upload again that I'm wondering if this is occurring on them as well.
The 2 I know this is happening are models of 3 gig and 1.5 gig that have grown to 23 gig and 10 gig. THey should only have 32 days of data in them. And I can only see 32 days of data.
Very strange to me. Maybe support will be able to answer what is going on. But this makes me rather paranoid. Since I only discovered this because I was getting out of memory errors on this. Which made me think I needed to buy a new capacity.