Forum Discussion
torsteinheggen
4 years agoFrequent Visitor
Removing "duplicates"
I have a data set with a lot of duplicates. Many of them, I can remove in Power Query with the remove duplicate function. But I am having great difficulties with names. The names are identical, but i...
- Anonymous4 years ago
Hi torsteinheggen - You can use the following approach, but I am not sure if this works if you want to keep the First Name, Surname result. The Remove Duplicates will keep the first available.
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCskvKi5JzcxT8EhNT0/NU4rViVaCMBVgUmAxl8SiyhwFn8q85AwwH8xSAIsqxcYCAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]), #"Added Custom" = Table.AddColumn( Source, "Custom", each Text.Combine( List.Sort( Text.ToList( Text.Lower( [Column1] ) ) ) ) ), #"Removed Duplicates" = Table.Distinct(#"Added Custom", {"Custom"}), #"Removed Columns" = Table.RemoveColumns(#"Removed Duplicates",{"Custom"}) in #"Removed Columns"Many thanks
Daryl
torsteinheggen
4 years agoFrequent Visitor
Think I missed something with the Text.ToList part. I'll investigate how to actually remove the duplicates, so I keep the rows I want. My main problem was the Table.AddColumn-part, which you solved.
Thank you so much Daryl.