Forum Discussion
Data Duplication Issue in Daily Power BI Desktop Refresh
The problem is not with the dataflow structure, as it is keeping only the most up-to-date record. The issue is more like a cache problem in the .pbix file, because, as illustrated above, there is no longer a row without a value in the deletedat field in the dataflow. It seems like the pbix file does not remove the data when starting the daily refresh, so the records that were updated conflict with the old information.
Show the entire power query code. Which field is the basis for the range filters?
Here is the complete Power Query code. The base field for the filter is updatedat.
Updatedat is not a suitable field. Incremental Refresh needs something immutable like Created Date. Even then you have to refresh historical partitions if the row contents changes.