Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
5 years ago
Solved

I can't remove duplicates.

I cannot remove duplicates.  I tried changing to uppercase and then removing duplicates.  It says I have 12056 rows and 12054 distinct values.  I tried filtering null values through the python script...
Need Help
  • Anonymous's avatar
    Anonymous
    5 years ago

    Yes.  I used trim and it worked,

mahoneypat's avatar
mahoneypat
Microsoft Employee
5 years ago

The 1000 rows is just the preview.  The Remove Duplicates step works on the whole dataset when it loads.  Were you able to make a table visual that shows with values are duplicated?

 

Pat

 

Anonymous's avatar
Anonymous
Not applicable
5 years ago

I made the table.  However, I cannot delete the values.  I am currently just using a many to many relationship.  Since the two sets of duplicates are the same throughout the row, the many to many relationship functions the same as a one to many.

  • v-kelly-msft's avatar
    v-kelly-msft
    Community Support
    5 years ago

    Hi  Anonymous ,

     

    Have you changed the below setting to the entire data set in power query before removing duplicates?

     

     

    Best Regards,
    Kelly

    Did I answer your question? Mark my post as a solution!