Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
aleggore
Frequent Visitor

Duplicate Values

I am finding duplicate values in my dataset after I try to refresh. There are no duplicate values showing when I run my query independently or in Power Query before I load to PowerBI. Has anyone seen this happen before and know of any resolution? 

3 REPLIES 3
Anonymous
Not applicable

Hi   @aleggore ,

Can't tell without seeing your query or data. Maybe you're expecting an inner join but used another join type?

"There are no duplicate values showing when I run my query independently or in Power Query before I load to PowerBI. "

 

The query editor only loads the first 1,000 records by default. You can force it to load all, then you'll get the error in the editor before closing.

 

you can test for duplicates

= Table.IsDistinct( table )

= Table.IsDistinct( table, { "pk_column1", "pk_column2" } )

or remove duplicates.

= Table.Distinct( table )

= Table.Distinct( table, { "pk_column1", "pk_column2" } ) 

 

It might have to do with Power BI's case-insensitivity:Power BI And Case Sensitivity 

For testing, you could add an index column (in Power Query) to check how the affected rows look like before and after loading.

 

If it doesn't help, please provide more details with the error.

 

 Best Regards
Community Support Team _ Polly

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

 

 

@aleggore

Turns out I was creating columns in my dataset in PowerBI that I should have been creating in Power Query. Making these changes, removed this duplicate situation. It was very odd. I'm a pretty new user so I'm still working through all the kinks 🙂

Greg_Deckler
Super User
Super User

@aleggore Extremely vague. Power Query only loads the first 1000 rows as preview. Are you appending some query to an existing table? Also, Power Query and DAX have different rules for what they consider duplicates. For example, DAX doesn't care about case while Power Query does. So, need more to go on in terms of what is going on.

 

Sorry, having trouble following, can you post sample data as text and expected output?
Not really enough information to go on, please first check if your issue is a common issue listed here: https://community.powerbi.com/t5/Community-Blog/Before-You-Post-Read-This/ba-p/1116882

Also, please see this post regarding How to Get Your Question Answered Quickly: https://community.powerbi.com/t5/Community-Blog/How-to-Get-Your-Question-Answered-Quickly/ba-p/38490

The most important parts are:
1. Sample data as text, use the table tool in the editing bar
2. Expected output from sample data
3. Explanation in words of how to get from 1. to 2.


Follow on LinkedIn
@ me in replies or I'll lose your thread!!!
Instead of a Kudo, please vote for this idea
Become an expert!: Enterprise DNA
External Tools: MSHGQM
YouTube Channel!: Microsoft Hates Greg
Latest book!:
The Definitive Guide to Power Query (M)

DAX is easy, CALCULATE makes DAX hard...

Helpful resources

Announcements
July 2024 Power BI Update

Power BI Monthly Update - July 2024

Check out the July 2024 Power BI update to learn about new features.

July Newsletter

Fabric Community Update - July 2024

Find out what's new and trending in the Fabric Community.