Remove Duplicates removing non duplicates!!

[dniedrauer]

Vessel NameVesselChemicalCustomerVessel_Chemical

Dawn Princess	9103996	Air Cooler Cleaner	Princess-T	9103996Air Cooler Cleaner
Dawn Princess	9103996	Aquabreak	Princess-T	9103996Aquabreak
Dawn Princess	9103996	Autotreat	Princess-T	9103996Autotreat
Dawn Princess	9103996	Boiler Coagulant	Princess-T	9103996Boiler Coagulant
Dawn Princess	9103996	Bromine	Princess-P	9103996Bromine
Dawn Princess	9103996	Carbonclean	Princess-T	9103996Carbonclean
Dawn Princess	9103996	Chlorine	Princess-P	9103996Chlorine
Dawn Princess	9103996	Cleanbreak	Princess-T	9103996Cleanbreak
Dawn Princess	9103996	Corrosion Inhibitor	Princess-T	9103996Corrosion Inhibitor
Dawn Princess	9103996	Deck Clean Np	Princess-T	9103996Deck Clean Np
Dawn Princess	9103996	Descaler	Princess-T	9103996Descaler
Dawn Princess	9103996	Electrosolv	Princess-T	9103996Electrosolv
Dawn Princess	9103996	Enviroclean	Princess-T	9103996Enviroclean
Dawn Princess	9103996	Hydrochloric Acid/Muriatic Acid	Princess-P	9103996Hydrochloric Acid/Muriatic Acid
Dawn Princess	9103996	Metal Brite	Princess-T	9103996Metal Brite
Dawn Princess	9103996	Oxygen Scavenger Plus	Princess-T	9103996Oxygen Scavenger Plus
Dawn Princess	9103996	Ph Increaser	Princess-P	9103996Ph Increaser
Dawn Princess	9103996	Pool & Spa Hardness	Princess-P	9103996Pool & Spa Hardness
Dawn Princess	9103996	Pool Clarifier	Princess-P	9103996Pool Clarifier
Dawn Princess	9103996	Teak Renewer	Princess-T	9103996Teak Renewer
Dawn Princess	9103996	Evaporator Treatment	Princess-T	9103996Evaporator Treatment

The top table shows what I've done. I've built a composite column in the far right and then I remove duplicates. 

Vessel NameVesselChemicalCustomerVessel_Chemical

Dawn Princess	9103996	Air Cooler Cleaner	Princess-T	9103996Air Cooler Cleaner
Dawn Princess	9103996	Boiler Coagulant	Princess-T	9103996Boiler Coagulant
Dawn Princess	9103996	Carbonclean	Princess-T	9103996Carbonclean
Dawn Princess	9103996	Deck Clean Np	Princess-T	9103996Deck Clean Np
Dawn Princess	9103996	Descaler	Princess-T	9103996Descaler
Dawn Princess	9103996	Enviroclean	Princess-T	9103996Enviroclean
Dawn Princess	9103996	Pool & Spa Hardness	Princess-P	9103996Pool & Spa Hardness
Dawn Princess	9103996	Pool Clarifier	Princess-P	9103996Pool Clarifier

Now, as you can see, there are a number of items missing that aren't duplicates. And it's just for this vessel (as far as I know). No idea what's going on, but this is bizarre and aggravating. 

[MattAllington]

. Which columns are you selecting before removing duplicates? 

That is what determines the removal. 

[dniedrauer]

Vessel is the numbered columns; I am selecting the concatenated column to the right to remove duplicates (I know I could just select multiple columns, but I also need that column for another purpose). 

I also tried removing duplicates prior to creating that custom column over to the far right, but it didn't remove all duplicates. 

[v-chuncz-msft]

@dniedrauer,

That sounds impossible. You may check the Advanced Editor. If convenient, show us the .pbix file.

[dniedrauer · Accepted solution]

Here's what the issue was. 

This table was comprised of a lookup table made by mashing two different data sets together; unfortunately, the vessel id (the number) was assigned to one name in one table and another name in the other table. So removing duplicates in terms of the id number removed many of the line allocations by name, even though the total by id number was correct. I did a replace of the incorrect name with the correct name, and now both the allocation and the consumption are correct. 

[MattAllington]

You should log a bug then