Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
SLI
Helper IV
Helper IV

Problem with incremental refresh on power bi premium

Hi everyone
I have a problem with incremental refresh on power bi premium.
I have set an incremental refresh on my power bi desktop (the aim is to refresh data of 2 last days) .
on power bi desktop, all works correctly but when I publish, the data is duplicated (for example on 2019-07-01 the amount of sales is duplicated by, 2)
Have you an idea about this problem?
The refresh of data is done every 30minute and the data is duplicated by 2 every time.

Thanks in advance for your help
SLI
1 ACCEPTED SOLUTION
Anonymous
Not applicable

hi @SLI 

Do you have a specific date column defined as LastUpdated date and specified the incremental policy in a the relevant table? Also make sure you have loaded the hisotrical data and refresh rows in the last ( XX) days.  detect data changes also critical to LastUpdatedDate changes. 

View solution in original post

13 REPLIES 13
hasenclever
Helper II
Helper II

That last comment from @oscarmqz  IS THE SOLUTION. Thanks

oscarmqz
Advocate II
Advocate II

The duplicated data is because of how you set up the filter. It as to be >= RangeStart and < RangeEnd. If you set >= and <= that's when you get dups

Thank you. 

@oscarmqz Thank you!

Anonymous
Not applicable

Thanks @oscarmqz , that fixed my issue.

Anonymous
Not applicable

Hi thanks for your reply
I am based on this article to set up the incremental refresh
Any one encountered the same problem??

Thanks

Hi, I have the same problem, were you able to fix this? I have a single INSERT_DT column which I am using for my RangeStart and RangeEnd filter, my settings are keep last 2 years of data and refresh for the last 2 days. It runs once a day, but all the daily snapshots are getting duplicated

Hi @oscarmqz 

 

yes my problem was resolved by adding a new column (dateLastUpdate) that i used for my RangeStart and RangeEnd filter

this column isn't updated in the source data, it shows only the last date of the update

 

please refer to this documentation

https://docs.microsoft.com/en-us/power-bi/service-premium-incremental-refresh#filter-date-column-upd...

 

regards 

Anonymous
Not applicable

hi @SLI 

How did you generate the "dataLastUpdate" column? I am having issues with duplicated values and my date.time column is comign from source system.

 

Thanks

Hi @Anonymous 

 

Sorry for the delay of my answer i've created a new column named "DateOfUpdate" and update this column with the datetime of the insertion of data

This solved my problem

the column has been added directly on the datawarehouse and data updated by SSIS

 

best regards

Hi,

 

i know this is an older thread, but when you say you created a dateLastUpdate column and that it is not updated in the source, how exactly are you generating that column?  Also, what column are you using for the incremental filters?

 

thanks

Scott

Anonymous
Not applicable

hi @SLI 

Do you have a specific date column defined as LastUpdated date and specified the incremental policy in a the relevant table? Also make sure you have loaded the hisotrical data and refresh rows in the last ( XX) days.  detect data changes also critical to LastUpdatedDate changes. 

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

February 2024 Update Carousel

Power BI Monthly Update - February 2024

Check out the February 2024 Power BI update to learn about new features.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.

Top Solution Authors