Remove duplicates doesn't work

[BI-1294]

Hi all,

I'm facing a very common problem regarding the remove duplicates function in power query. If I try to remove duplicates from the column "username", not all the duplicates are removed (Link for the csv file - open in with Firefox)

I've tried using trim, clean, remove blanks, remove errors transformations before doing the eliminate duplicates, but it doesn't work.

Is there anyone you can help me?

In this moment I fixed the problem via dax, but I would prefer to do it in power query and not in dax.

Thanks a lot for your help

[AlexisOlson]

Can you share the M code you are using (you can grab it from the Advanced Editor)?

[BI-1294]

Yes, of course. Here below the M code.

An example of user that this code can't remove is "Jvn_91" and I couldn't figure it out why.

[AlexisOlson]

Hmm. That's pretty odd. Any chance you can share the CSV or an edited version of it so we can try to reproduce your result?

On the off-chance this is just a caching thing, I recommend hitting Refresh Preview again.

[BI-1294]

I just tried to hit Refresh Preview, but it doesn't work. That's really strange because it always worked in the past.

Here there is the csv file (Link for the csv file - open it with Firefox because the other browers can't open it)

[AlexisOlson · Accepted solution]

Looking at your data, these rows aren't duplicates since Power Query is case-sensitive.

If you want these to be considered the same, I'd recommend adding a step to transform to all lowercase before removing duplicates.

[BI-1294]

Actually, I don't see the difference "j" in this case. If I see my data they are both capitalized. How do you see them differently from me?

Here there is another example:

[AlexisOlson]

You are looking at the table in the Data pane rather than the query editor like I was.

I'm guessing the engine selected a single representation to show since DAX doesn't inherently distinguish case like M does.

[BI-1294]

Thanks a lot, I didn't notice that the two have different initials 😅