Remove Duplicates not working in power query

[Somu_Mohan]

Hello,

I am not able to remove duplicates from an insured name column. I have tried clean and trim. Checked by loading it to excel as i was not able to create relationships in DAX, I see that this is not due to case sensitive. Surprisingly if copied this result make it a new table and new query it removes the duplicates. Normal excel countif gave me the count 2 in column1, meaning excel also sees it as duplicates. Why is PQ posing a issue here?

[Somu_Mohan]

Hi Stephen,

Thanks for your support, the attached screenshot initially was arrived at by carrying out a remove duplicate action(where insured name was the only column present). Initially I had 600+ insured names. The "column1" is a excel column where I applied countif, which was resulting in value = 2. As this "Insured Name" columns was being arrived after tramsforming, cleaning then referencing a big dataset, the steps you suggested in part - 2 was taking too long. Unfortunately it was not feasible for me.

The solution that worked, I applied the below steps to my master data as follows.

What I am not able to understand is that, the result with duplicates when goes in as new query it removes them perfectly. so why not in the first instance? why is Uppercased making a difference?

[v-stephen-msft]

Hi @Somu_Mohan ,

When there're only two column in your table, you can successfully remove duplicates.

It removes duplicate rows based on the removal of an entire row, not a column of duplicate rows.

For example, in the following data, if you want to remove duplicate rows in the Insured Name column.

Here's the solution.

1.Group by.

2.Add an index column.

3.Expand it and filter 1.

Best Regards,

Stephen Tao

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.