Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Be one of the first to start using Fabric Databases. View on-demand sessions with database experts and the Microsoft product team to learn just how easy it is to get started. Watch now

Reply
SLI
Helper IV
Helper IV

Problem with incremental refresh on power bi premium

Hi everyone
I have a problem with incremental refresh on power bi premium.
I have set an incremental refresh on my power bi desktop (the aim is to refresh data of 2 last days) .
on power bi desktop, all works correctly but when I publish, the data is duplicated (for example on 2019-07-01 the amount of sales is duplicated by, 2)
Have you an idea about this problem?
The refresh of data is done every 30minute and the data is duplicated by 2 every time.

Thanks in advance for your help
SLI
1 ACCEPTED SOLUTION
Anonymous
Not applicable

hi @SLI 

Do you have a specific date column defined as LastUpdated date and specified the incremental policy in a the relevant table? Also make sure you have loaded the hisotrical data and refresh rows in the last ( XX) days.  detect data changes also critical to LastUpdatedDate changes. 

View solution in original post

13 REPLIES 13
hasenclever
Helper II
Helper II

That last comment from @oscarmqz  IS THE SOLUTION. Thanks

oscarmqz
Advocate II
Advocate II

The duplicated data is because of how you set up the filter. It as to be >= RangeStart and < RangeEnd. If you set >= and <= that's when you get dups

Thank you. 

@oscarmqz Thank you!

Anonymous
Not applicable

Thanks @oscarmqz , that fixed my issue.

Anonymous
Not applicable

Hi thanks for your reply
I am based on this article to set up the incremental refresh
Any one encountered the same problem??

Thanks

Hi, I have the same problem, were you able to fix this? I have a single INSERT_DT column which I am using for my RangeStart and RangeEnd filter, my settings are keep last 2 years of data and refresh for the last 2 days. It runs once a day, but all the daily snapshots are getting duplicated

Hi @oscarmqz 

 

yes my problem was resolved by adding a new column (dateLastUpdate) that i used for my RangeStart and RangeEnd filter

this column isn't updated in the source data, it shows only the last date of the update

 

please refer to this documentation

https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh#filter-date-column-upd...

 

regards 

Anonymous
Not applicable

hi @SLI 

How did you generate the "dataLastUpdate" column? I am having issues with duplicated values and my date.time column is comign from source system.

 

Thanks

Hi @Anonymous 

 

Sorry for the delay of my answer i've created a new column named "DateOfUpdate" and update this column with the datetime of the insertion of data

This solved my problem

the column has been added directly on the datawarehouse and data updated by SSIS

 

best regards

Hi,

 

i know this is an older thread, but when you say you created a dateLastUpdate column and that it is not updated in the source, how exactly are you generating that column?  Also, what column are you using for the incremental filters?

 

thanks

Scott

Anonymous
Not applicable

hi @SLI 

Do you have a specific date column defined as LastUpdated date and specified the incremental policy in a the relevant table? Also make sure you have loaded the hisotrical data and refresh rows in the last ( XX) days.  detect data changes also critical to LastUpdatedDate changes. 

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

November Carousel

Fabric Community Update - November 2024

Find out what's new and trending in the Fabric Community.

Dec Fabric Community Survey

We want your feedback!

Your insights matter. That’s why we created a quick survey to learn about your experience finding answers to technical questions.