cancel
Showing results for 
Search instead for 
Did you mean: 

Fabric is Generally Available. Browse Fabric Presentations. Work towards your Fabric certification with the Cloud Skills Challenge.

Reply
zenisekd
Helper III
Helper III

"Remove duplicates" removes unique values in the format of whole number

Hi...

my issue is that I have a table, where the column ID is in the format of whole number (int), right from the beginning, without any transformations. The table has around 25k rows. 
When I use "remove duplicate rows" on ID column, it deletes, along with duplicates, certain unique numbers eg. 23682. 

 

zenisekd_2-1627306001302.png

When I revert the steps (first filter the number and then remove duplicates), the number is there ...

 

zenisekd_0-1627305951225.png

zenisekd_1-1627305963554.png

Can somebody tell me what the heck is going on?

I have read in different places that sometimes it helps to first convert the column to text, then trim, do upper or lower case, then remove duplicate rows and then convert it back to a number. This doesn't work for me, however...

1 ACCEPTED SOLUTION
v-kelly-msft
Community Support
Community Support

Hi  @zenisekd ,

 

After checking,I couldnt reproduce your senario,as it seems fine here:

vkellymsft_2-1627464824724.png

 

vkellymsft_1-1627464812930.png

 

So pls check the blog below to see whether you could find a solution:

https://radacad.com/remove-duplicate-doesnt-work-in-power-query-for-power-bi-here-is-the-solution

 

Best Regards,
Kelly

Did I answer your question? Mark my post as a solution!

View solution in original post

6 REPLIES 6
j_ocean
Resolver I
Resolver I

Are you sure the removed duplicates is pointing at the ID column? That's caught me before, pointing at some kind of title where it may coincidentally be the same across two entries.

v-kelly-msft
Community Support
Community Support

Hi  @zenisekd ,

 

After checking,I couldnt reproduce your senario,as it seems fine here:

vkellymsft_2-1627464824724.png

 

vkellymsft_1-1627464812930.png

 

So pls check the blog below to see whether you could find a solution:

https://radacad.com/remove-duplicate-doesnt-work-in-power-query-for-power-bi-here-is-the-solution

 

Best Regards,
Kelly

Did I answer your question? Mark my post as a solution!

Any ETA for this bug to be fixed?

I guess no ETA for a fix.

samdthompson
Memorable Member
Memorable Member

Hello, i have had this too. I am not sure of the techincal reason but i got around it by prefixing all the text values of whole numbers with a z then doing the duplicate removal, removing the z's and converting back to a whole number. A bit convoluted but it worked for me.

 

 

// if this is a solution please mark as such. Kudos always appreciated.

Got same problem here. Removing duplicates on a Whole Number column was in fact removing thousands of unique values. Changing type to text before removing duplicates and then back to Int has worked fine.

JoseRamon_0-1692256816126.png

 

Helpful resources

Announcements
PBI November 2023 Update Carousel

Power BI Monthly Update - November 2023

Check out the November 2023 Power BI update to learn about new features.

Community News

Fabric Community News unified experience

Read the latest Fabric Community announcements, including updates on Power BI, Synapse, Data Factory and Data Activator.

Power BI Fabric Summit Carousel

The largest Power BI and Fabric virtual conference

130+ sessions, 130+ speakers, Product managers, MVPs, and experts. All about Power BI and Fabric. Attend online or watch the recordings.

Top Solution Authors
Top Kudoed Authors