Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The Power BI Data Visualization World Championships is back! Get ahead of the game and start preparing now! Learn more

Reply
purple_SP
Helper I
Helper I

Finding duplicate lists where the values are in different orders

Hello, I am working with some data in the following type of format where column 3 is a list of values:

 

Column 1 Column 2 Column 3 
John Smith A,B
Sarah Jones B,C
Jane Wood A
Kyle Ford A,B,C
JohnSmithB,A

 

I would like to remove duplicates in this table. In column 3, the first and last rows are duplicated, but the values in the list are in different orders. Is there a way to identify and remove these duplicates? Ideally I would like the final result to look like this:

 

Column 1 Column 2 Column 3
John Smith A,B
Sarah Jones B,C
Jane Wood A
Kyle Ford A,B,C

 

Any help would be much appreciated.

1 ACCEPTED SOLUTION
wdx223_Daniel
Super User
Super User

NewStep=Table.Distinct(Table.TransformColumns(PreviousStepName,{"Column 3",each Text.Combine(List.Sort(Text.Split(_,",")),",")}))

View solution in original post

2 REPLIES 2
6677028
Frequent Visitor

You check 

let

    Source = Excel.CurrentWorkbook(){[Name="Table3"]}[Content],

    #"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),

    #"Sorted Rows" = Table.Sort(#"Promoted Headers",{{"Col1", Order.Ascending}}),

    #"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 1, 1, Int64.Type),

    #"Added Index1" = Table.AddIndexColumn(#"Added Index", "Index.1", 0, 1, Int64.Type),

    #"Merged Queries" = Table.NestedJoin(#"Added Index1", {"Index"}, #"Added Index1", {"Index.1"}, "Added Index1", JoinKind.FullOuter),

    #"Sorted Rows1" = Table.Sort(#"Merged Queries",{{"Index", Order.Ascending}}),

    #"Expanded Added Index1" = Table.ExpandTableColumn(#"Sorted Rows1", "Added Index1", {"Col1"}, {"Added Index1.Col1"}),

    #"Added Custom" = Table.AddColumn(#"Expanded Added Index1", "Custom", each if [Col1]=[Added Index1.Col1] then null else [Col1]),

    #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index", "Index.1", "Added Index1.Col1", "Col1"}),

    #"Reordered Columns" = Table.ReorderColumns(#"Removed Columns",{"Custom", "Col2", "Col3"}),

    #"Filtered Rows" = Table.SelectRows(#"Reordered Columns", each ([Custom] <> null))

in

    #"Filtered Rows"

wdx223_Daniel
Super User
Super User

NewStep=Table.Distinct(Table.TransformColumns(PreviousStepName,{"Column 3",each Text.Combine(List.Sort(Text.Split(_,",")),",")}))

Helpful resources

Announcements
Power BI DataViz World Championships

Power BI Dataviz World Championships

The Power BI Data Visualization World Championships is back! Get ahead of the game and start preparing now!

December 2025 Power BI Update Carousel

Power BI Monthly Update - December 2025

Check out the December 2025 Power BI Holiday Recap!

FabCon Atlanta 2026 carousel

FabCon Atlanta 2026

Join us at FabCon Atlanta, March 16-20, for the ultimate Fabric, Power BI, AI and SQL community-led event. Save $200 with code FABCOMM.