Duplicate Values

[aleggore]

I am finding duplicate values in my dataset after I try to refresh. There are no duplicate values showing when I run my query independently or in Power Query before I load to PowerBI. Has anyone seen this happen before and know of any resolution? 

[v-rongtiep-msft]

Hi   @aleggore ,

Can't tell without seeing your query or data. Maybe you're expecting an inner join but used another join type?

"There are no duplicate values showing when I run my query independently or in Power Query before I load to PowerBI. "

The query editor only loads the first 1,000 records by default. You can force it to load all, then you'll get the error in the editor before closing.

you can test for duplicates

= Table.IsDistinct( table )

= Table.IsDistinct( table, { "pk_column1", "pk_column2" } )

or remove duplicates.

= Table.Distinct( table )

= Table.Distinct( table, { "pk_column1", "pk_column2" } ) 

It might have to do with Power BI's case-insensitivity:Power BI And Case Sensitivity 

For testing, you could add an index column (in Power Query) to check how the affected rows look like before and after loading.

If it doesn't help, please provide more details with the error.

 Best Regards
Community Support Team _ Polly

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

@aleggore

[aleggore]

Turns out I was creating columns in my dataset in PowerBI that I should have been creating in Power Query. Making these changes, removed this duplicate situation. It was very odd. I'm a pretty new user so I'm still working through all the kinks 🙂

[Greg_Deckler]

@aleggore Extremely vague. Power Query only loads the first 1000 rows as preview. Are you appending some query to an existing table? Also, Power Query and DAX have different rules for what they consider duplicates. For example, DAX doesn't care about case while Power Query does. So, need more to go on in terms of what is going on.

Sorry, having trouble following, can you post sample data as text and expected output?
Not really enough information to go on, please first check if your issue is a common issue listed here: https://community.powerbi.com/t5/Community-Blog/Before-You-Post-Read-This/ba-p/1116882

Also, please see this post regarding How to Get Your Question Answered Quickly: https://community.powerbi.com/t5/Community-Blog/How-to-Get-Your-Question-Answered-Quickly/ba-p/38490

The most important parts are:
1. Sample data as text, use the table tool in the editing bar
2. Expected output from sample data
3. Explanation in words of how to get from 1. to 2.