Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Reply
Anonymous
Not applicable

Duplicate Value error in incremental refresh

Hi Team,

 

I am getting below error , while performing incremental refresh. There are no duplicate values on SQL side. This happens when there is update in the table. 

jyotirpawar_0-1620811059227.png

 

I have set up incremental refresh based on the Last_Updated_Date (For RangeStart and RangeEnd), and same column is used for Detect Changes. Is that causing this error? 

 

I have gone through multiple threads for this issue, but none of them are helping me because for me there are actually no duplicates.

 

Thank you in advance!

 

Regards,

Jyoti Pawar

3 REPLIES 3
Anonymous
Not applicable

Hi  @Anonymous  ,

 

It may be the "Auto-detect relationship" function that is turned on by default. You can check "Manage Relationships" to see if you have connected some tables without your knowledge, and automatically created several very strange relationships. When the only column is duplicated, the automatically created relationships now become "many-to-many. You can disable these unnecessarily created relationships, and turn off the "automatically detect relationships" function.

 

Best Regards,

Liu Yang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Anonymous
Not applicable

Did changes as you said. But Still im getting the same error. Switched Off incremntal refresh for the table which is throwing that error then it got resolved. But i need to turn on the Incremental refresh for future data. 
Will be appreciated if any one provides solution

ibarrau
Super User
Super User

Hi. A table with a one to many relation cannot have duplicates. Now I'm wondering, do you have a star schema data model? because if you do, your fact won't be a 1 in the relationships so duplications won't matter. The thing is that if you want an incremental dimension you might want to consider reading more about Slowly Changing Dimnesion to handle unique keys. Incremental refresh is usually recommended for fact tables that won't be on the one side of the relationship.

Consider it might happen that you the engine finds a duplicate value that your source is not showing. Consider that any space or lower/upper case that the source is considering different, it will be the same in the data model on Power Bi.

I hope that helps.


If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Happy to help!

LaDataWeb Blog

Helpful resources

Announcements
July 2025 community update carousel

Fabric Community Update - July 2025

Find out what's new and trending in the Fabric community.

July PBI25 Carousel

Power BI Monthly Update - July 2025

Check out the July 2025 Power BI update to learn about new features.

Join our Fabric User Panel

Join our Fabric User Panel

This is your chance to engage directly with the engineering team behind Fabric and Power BI. Share your experiences and shape the future.