Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Join us for an expert-led overview of the tools and concepts you'll need to become a Certified Power BI Data Analyst and pass exam PL-300. Register now.

Reply
zenisekd
Super User
Super User

"Remove duplicates" removes unique values in the format of whole number

Hi...

my issue is that I have a table, where the column ID is in the format of whole number (int), right from the beginning, without any transformations. The table has around 25k rows. 
When I use "remove duplicate rows" on ID column, it deletes, along with duplicates, certain unique numbers eg. 23682. 

 

zenisekd_2-1627306001302.png

When I revert the steps (first filter the number and then remove duplicates), the number is there ...

 

zenisekd_0-1627305951225.png

zenisekd_1-1627305963554.png

Can somebody tell me what the heck is going on?

I have read in different places that sometimes it helps to first convert the column to text, then trim, do upper or lower case, then remove duplicate rows and then convert it back to a number. This doesn't work for me, however...

1 ACCEPTED SOLUTION
v-kelly-msft
Community Support
Community Support

Hi  @zenisekd ,

 

After checking,I couldnt reproduce your senario,as it seems fine here:

vkellymsft_2-1627464824724.png

 

vkellymsft_1-1627464812930.png

 

So pls check the blog below to see whether you could find a solution:

https://radacad.com/remove-duplicate-doesnt-work-in-power-query-for-power-bi-here-is-the-solution

 

Best Regards,
Kelly

Did I answer your question? Mark my post as a solution!

View solution in original post

7 REPLIES 7
VVA
New Member

I have the same issue but just with the regular OOB function. I have a list of names and I want to remove the duplicates. It seems that the function removes unique values (returns 19 'unique', when I know for a fact the unique names are around 75!). How to solve this issue? Why is it happening?😩

j_ocean
Helper V
Helper V

Are you sure the removed duplicates is pointing at the ID column? That's caught me before, pointing at some kind of title where it may coincidentally be the same across two entries.

v-kelly-msft
Community Support
Community Support

Hi  @zenisekd ,

 

After checking,I couldnt reproduce your senario,as it seems fine here:

vkellymsft_2-1627464824724.png

 

vkellymsft_1-1627464812930.png

 

So pls check the blog below to see whether you could find a solution:

https://radacad.com/remove-duplicate-doesnt-work-in-power-query-for-power-bi-here-is-the-solution

 

Best Regards,
Kelly

Did I answer your question? Mark my post as a solution!

Any ETA for this bug to be fixed?

I guess no ETA for a fix.

samdthompson
Memorable Member
Memorable Member

Hello, i have had this too. I am not sure of the techincal reason but i got around it by prefixing all the text values of whole numbers with a z then doing the duplicate removal, removing the z's and converting back to a whole number. A bit convoluted but it worked for me.

 

 

// if this is a solution please mark as such. Kudos always appreciated.

Got same problem here. Removing duplicates on a Whole Number column was in fact removing thousands of unique values. Changing type to text before removing duplicates and then back to Int has worked fine.

JoseRamon_0-1692256816126.png

 

Helpful resources

Announcements
Join our Fabric User Panel

Join our Fabric User Panel

This is your chance to engage directly with the engineering team behind Fabric and Power BI. Share your experiences and shape the future.

June 2025 Power BI Update Carousel

Power BI Monthly Update - June 2025

Check out the June 2025 Power BI update to learn about new features.

June 2025 community update carousel

Fabric Community Update - June 2025

Find out what's new and trending in the Fabric community.