Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
5 years ago
Solved

I can't remove duplicates.

I cannot remove duplicates.  I tried changing to uppercase and then removing duplicates.  It says I have 12056 rows and 12054 distinct values.  I tried filtering null values through the python script...
Need Help
  • Anonymous's avatar
    Anonymous
    5 years ago

    Yes.  I used trim and it worked,

Anonymous's avatar
Anonymous
Not applicable
5 years ago

However, I still cannot remove them.  Power Query does not work and it has too many rows and won't let me filter.

mahoneypat's avatar
mahoneypat
Microsoft Employee
5 years ago

The 1000 rows is just the preview.  The Remove Duplicates step works on the whole dataset when it loads.  Were you able to make a table visual that shows with values are duplicated?

 

Pat

 

  • Anonymous's avatar
    Anonymous
    Not applicable
    5 years ago

    I made the table.  However, I cannot delete the values.  I am currently just using a many to many relationship.  Since the two sets of duplicates are the same throughout the row, the many to many relationship functions the same as a one to many.

    • v-kelly-msft's avatar
      v-kelly-msft
      Community Support
      5 years ago

      Hi  Anonymous ,

       

      Have you changed the below setting to the entire data set in power query before removing duplicates?

       

       

      Best Regards,
      Kelly

      Did I answer your question? Mark my post as a solution!