Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
5 years ago
Solved

I can't remove duplicates.

I cannot remove duplicates.  I tried changing to uppercase and then removing duplicates.  It says I have 12056 rows and 12054 distinct values.  I tried filtering null values through the python script...
Need Help
  • Anonymous's avatar
    Anonymous
    5 years ago

    Yes.  I used trim and it worked,

mahoneypat's avatar
mahoneypat
Microsoft Employee
5 years ago

You've already done the uppercase thing, which gets a lot of people.  Do you have some values with hidden characters?  If so, try adding a trim step before your Remove Duplicates step.

 

Also, if not already, you can make a table visual with that column and add it again to get the count of rows, then sort by that to see which ones have >1 and are causing your issue.

 

Pat

Pat

 

Anonymous's avatar
Anonymous
Not applicable
5 years ago

I found the duplicates.  I think deleting duplicates isn't working because it only deletes for the top 1000 rows in power query.

  • mussaenda's avatar
    mussaenda
    Community Champion
    5 years ago

    Hi Anonymous ,

     

    Removing duplicates does not only apply to top 1000 rows. it applies to your whole table. top 1000 rows is only for preview. Try checking other columns those might be the cause of data duplicate.

     

    Hope this helps.