Forum Discussion
I can't remove duplicates.
I cannot remove duplicates. I tried changing to uppercase and then removing duplicates. It says I have 12056 rows and 12054 distinct values. I tried filtering null values through the python script I use to bring my data in. I tried filtering out empty cells in power query. I exported a table in BI to excel and it says there are 12054 rows.
10 Replies
You've already done the uppercase thing, which gets a lot of people. Do you have some values with hidden characters? If so, try adding a trim step before your Remove Duplicates step.
Also, if not already, you can make a table visual with that column and add it again to get the count of rows, then sort by that to see which ones have >1 and are causing your issue.
Pat
Pat
Hi Anonymous ,
Removing duplicates does not only apply to top 1000 rows. it applies to your whole table. top 1000 rows is only for preview. Try checking other columns those might be the cause of data duplicate.
Hope this helps.
The 1000 rows is just the preview. The Remove Duplicates step works on the whole dataset when it loads. Were you able to make a table visual that shows with values are duplicated?
Pat
Hi Anonymous ,
Is your issue solved now?
Best Regards,
KellyDid I answer your question? Mark my post as a solution!
Hi Anonymous ,
Glad to hear it,thanks for sharing your solution,could you pls make the reply as answered to let more people find the solution?
Best Regards,
KellyDid I answer your question? Mark my post as a solution!
