Problem with incremental refresh on power bi premium

[SLI]

Hi everyone
I have a problem with incremental refresh on power bi premium.
I have set an incremental refresh on my power bi desktop (the aim is to refresh data of 2 last days) .
on power bi desktop, all works correctly but when I publish, the data is duplicated (for example on 2019-07-01 the amount of sales is duplicated by, 2)
Have you an idea about this problem?
The refresh of data is done every 30minute and the data is duplicated by 2 every time.

Thanks in advance for your help
SLI

[hasenclever]

That last comment from @oscarmqz  IS THE SOLUTION. Thanks

[oscarmqz]

The duplicated data is because of how you set up the filter. It as to be >= RangeStart and < RangeEnd. If you set >= and <= that's when you get dups

[sujith31]

Thank you. 

[Janeilson]

@oscarmqz Thank you!

[Anonymous]

Thanks @oscarmqz , that fixed my issue.

[Anonymous]

hi @SLI 

please refer this documentation for configuration

https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh#refresh-ranges

Important

https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh#filter-date-column-upd...

Thanks

[SLI]

Hi thanks for your reply
I am based on this article to set up the incremental refresh
Any one encountered the same problem??

Thanks

[oscarmqz]

Hi, I have the same problem, were you able to fix this? I have a single INSERT_DT column which I am using for my RangeStart and RangeEnd filter, my settings are keep last 2 years of data and refresh for the last 2 days. It runs once a day, but all the daily snapshots are getting duplicated

[SLI]

Hi @oscarmqz 

yes my problem was resolved by adding a new column (dateLastUpdate) that i used for my RangeStart and RangeEnd filter

this column isn't updated in the source data, it shows only the last date of the update

please refer to this documentation

https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh#filter-date-column-upd...

regards 

[Anonymous]

hi @SLI 

How did you generate the "dataLastUpdate" column? I am having issues with duplicated values and my date.time column is comign from source system.

Thanks

[SLI]

Hi @Anonymous 

Sorry for the delay of my answer i've created a new column named "DateOfUpdate" and update this column with the datetime of the insertion of data

This solved my problem

the column has been added directly on the datawarehouse and data updated by SSIS

best regards

[scabral]

Hi,

i know this is an older thread, but when you say you created a dateLastUpdate column and that it is not updated in the source, how exactly are you generating that column?  Also, what column are you using for the incremental filters?

thanks

Scott

[Anonymous · Accepted solution]

hi @SLI 

Do you have a specific date column defined as LastUpdated date and specified the incremental policy in a the relevant table? Also make sure you have loaded the hisotrical data and refresh rows in the last ( XX) days.  detect data changes also critical to LastUpdatedDate changes.