Forum Discussion
Incremental refresh on api considering last modified date
Incremental refresh expects immutable data. What you are looking for is differential refresh.
One workaround is to make the refresh scope large enough so you can shoehorn your data into the incremental refresh partition paradigm. Ask yourself what the average maximum period of time is that a issue change can still make a difference in your business process, and then set that as the change period for the incremental refresh (for example "last updated in the last three months".
Incremental refresh expects immutable data. What you are looking for is differential refresh.
One workaround is to make the refresh scope large enough so you can shoehorn your data into the incremental refresh partition paradigm. Ask yourself what the average maximum period of time is that a issue change can still make a difference in your business process, and then set that as the change period for the incremental refresh (for example "last updated in the last three months".
Thank you so much for your reply. Much appreciated.
Unfortunately, an issue might be created even 2 years ago and get update today. This might happen mainly due to many automations doing bulk updates.
So using the workaround you suggested, will lead still to big partitions, something that I wanted to avoid.
I hoped the Power BI incremental refresh mechanism would allow me to avoid the need for a more sophisticated DWH. Seems it can't be done in this case 🙂
Thank you for your help. I've learnt something important!
P.S. I don't remember seeing the assumption of "immutable data" in the documentation. I guess it should be mentioned more explicitly...