Re: Duplicate Value

congDA · ‎07-14-2022

Evething is okay until yesterday but I got an refresher error this morning. It said that FACT_ORDERS table contains a duplicate value, so I go to the database and write a simple query to check: there are no duplicate values showing.

I found the solution in PBI Community (use Remove Duplicates in Power Query) but I really want to know what caused this issue. Does anyone know?

Thanks!

otravers · ‎07-14-2022

Are you using incremental refresh on this table?

------------------------------------------------
1. How to get your question answered quickly - good questions get good answers!
2. Learning how to fish > being spoon-fed without active thinking.
3. Please accept as a solution posts that resolve your questions.
------------------------------------------------
BI Blog: Datamarts | RLS/OLS | Dev Tools | Languages | Aggregations | XMLA/APIs | Field Parameters | Custom Visuals

congDA · ‎07-15-2022

Yes, I am. Does it make this error? Could you explain for me? Thank you

otravers · ‎07-15-2022

Check that you're using < RangeEnd, not <= in your filter.

------------------------------------------------
1. How to get your question answered quickly - good questions get good answers!
2. Learning how to fish > being spoon-fed without active thinking.
3. Please accept as a solution posts that resolve your questions.
------------------------------------------------
BI Blog: Datamarts | RLS/OLS | Dev Tools | Languages | Aggregations | XMLA/APIs | Field Parameters | Custom Visuals

congDA · ‎07-20-2022

I'm using < RangeEnd 😭 I still don't know why this happened. Why is it related to RangEnd and incremental refresh?

otravers · ‎07-20-2022

Why improperly setting RangeStart/RangeEnd filters can lead to duplicates is documented here:

https://docs.microsoft.com/en-us/power-bi/connect-data/incremental-refresh-configure#filter-data

In your case it doesn't seem to be the root cause for your duplicates. If there are no duplicates in your source data then I'd doublecheck that I have nothing in my ETL (e.g. appending tables) that may cause these duplicates.

------------------------------------------------
1. How to get your question answered quickly - good questions get good answers!
2. Learning how to fish > being spoon-fed without active thinking.
3. Please accept as a solution posts that resolve your questions.
------------------------------------------------
BI Blog: Datamarts | RLS/OLS | Dev Tools | Languages | Aggregations | XMLA/APIs | Field Parameters | Custom Visuals