Forum Discussion
Incremental refresh on api considering last modified date
Incremental refresh expects immutable data. What you are looking for is differential refresh.
One workaround is to make the refresh scope large enough so you can shoehorn your data into the incremental refresh partition paradigm. Ask yourself what the average maximum period of time is that a issue change can still make a difference in your business process, and then set that as the change period for the incremental refresh (for example "last updated in the last three months".
karimm I landed here with the EXACT same issue with Jira and Updated dates. What did you end up doing?
All I can think of is to import my [Jira Issues] table using incremental refresh for the past 2 days, ralizing that there will be duplicate Issue Keys. So my table is basically a heap (instead of having Issue Key as the key), with old obsolete records.
Then, I would create a calculated {Issues} table in DAX using some fancy logic to get the row with the most recent updated timestamp per Issue Key. That would be the table used in reports/visuals.
Another hacky solution I thought about would be to use a Power BI datamart to just pull the data into a SQL format, whish would make it a bit easier (at least for me) to model for uniqueness while still geting 5-second refreshes.
The only "solution" is to monitor partition "truthiness" and periodically do full refreshes of these partitions if they become too corrupted.