Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The Power BI DataViz World Championships are on! With four chances to enter, you could win a spot in the LIVE Grand Finale in Las Vegas. Show off your skills.

Reply
congDA
Frequent Visitor

Duplicate Value

Evething is okay until yesterday but I got an refresher error this morning. It said that FACT_ORDERS table contains a duplicate value, so I go to the database and write a simple query to check: there are no duplicate values showing. 

 

I found the solution in PBI Community (use Remove Duplicates in Power Query) but I really want to know what caused this issue. Does anyone know?

 

Thanks!

 

Screenshot 2022-07-14 154911.png

5 REPLIES 5
otravers
Community Champion
Community Champion

Are you using incremental refresh on this table?

------------------------------------------------
1. How to get your question answered quickly - good questions get good answers!
2. Learning how to fish > being spoon-fed without active thinking.
3. Please accept as a solution posts that resolve your questions.
------------------------------------------------
BI Blog: Datamarts | RLS/OLS | Dev Tools | Languages | Aggregations | XMLA/APIs | Field Parameters | Custom Visuals

Yes, I am. Does it make this error? Could you explain for me? Thank you

otravers
Community Champion
Community Champion

Check that you're using < RangeEnd, not <= in your filter.

------------------------------------------------
1. How to get your question answered quickly - good questions get good answers!
2. Learning how to fish > being spoon-fed without active thinking.
3. Please accept as a solution posts that resolve your questions.
------------------------------------------------
BI Blog: Datamarts | RLS/OLS | Dev Tools | Languages | Aggregations | XMLA/APIs | Field Parameters | Custom Visuals

I'm using < RangeEnd 😭 I still don't know why this happened. Why is it related to RangEnd and incremental refresh?

otravers
Community Champion
Community Champion

Why improperly setting RangeStart/RangeEnd filters can lead to duplicates is documented here:

https://docs.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-configure#filter-data

 

In your case it doesn't seem to be the root cause for your duplicates. If there are no duplicates in your source data then I'd doublecheck that I have nothing in my ETL (e.g. appending tables) that may cause these duplicates.

------------------------------------------------
1. How to get your question answered quickly - good questions get good answers!
2. Learning how to fish > being spoon-fed without active thinking.
3. Please accept as a solution posts that resolve your questions.
------------------------------------------------
BI Blog: Datamarts | RLS/OLS | Dev Tools | Languages | Aggregations | XMLA/APIs | Field Parameters | Custom Visuals

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

Feb2025 Sticker Challenge

Join our Community Sticker Challenge 2025

If you love stickers, then you will definitely want to check out our Community Sticker Challenge!

Feb2025 NL Carousel

Fabric Community Update - February 2025

Find out what's new and trending in the Fabric community.