Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The Power BI Data Visualization World Championships is back! Get ahead of the game and start preparing now! Learn more

Reply
Anonymous
Not applicable

Remove Duplicate Rows Based on Value

In my data there are several duplicates of Record ID and Brand. The same combination fo Record ID and Brand can refer to different Inv IDs. I want to remove those cases Record ID / Brand combination where Inv ID is empty when for the same Record ID / Brand combination a row with an Inv ID exists. For cases where there is only a single row for a Record ID / Brand combination where Inv ID is empty, I want to keep that. Does anyone know how to do that? I'm stuck...

 

See the below example:

 

The table I have:

Record IDBrandInv ID
R1Brand1I1
R1Brand1I2
R1Brand1null
R2Brand2null
R3Brand3I3
R4Brand4null
R4Brand4I4

 

The table I'd like to return:

Record IDBrandInv ID
R1Brand1I1
R1Brand1I2
R2Brand2null
R3Brand3I3
R4Brand4I4
1 ACCEPTED SOLUTION
ronrsnfld
Super User
Super User

Group by Record ID, then process each subtable:

 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCjJU0lFyKkrMSwExPA2VYnUwBI2wCEKEjGBCRnAhY5gQiOFpDBE0gQmawNWhCHmaKMXGAgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Record ID" = _t, Brand = _t, #"Inv ID" = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{
        {"Record ID", type text}, {"Brand", type text}, {"Inv ID", type text}}),
    
    //Group by ID, then process each subtable
    #"Grouped Rows" = Table.Group(#"Changed Type", {"Record ID"}, {
        {"Rem Dups", (t)=>
            let 
                unique = Table.Distinct(t),
                #"Rem Nulls" = if Table.RowCount(unique) > 1
                    then Table.SelectRows(unique, each [Inv ID] <> null and [Inv ID] <> "") else unique
            in #"Rem Nulls",
                type table [Record ID=nullable text, Brand=nullable text, Inv ID=nullable text]}}),
    
    #"Expanded Rem Dups" = Table.ExpandTableColumn(#"Grouped Rows", "Rem Dups", {"Brand", "Inv ID"}, {"Brand", "Inv ID"})
in
    #"Expanded Rem Dups"

Data

ronrsnfld_0-1707340321890.png

Results

ronrsnfld_1-1707340359152.png

 

 

 

 

View solution in original post

2 REPLIES 2
Anonymous
Not applicable

Thank you so much, it works perfectly!

ronrsnfld
Super User
Super User

Group by Record ID, then process each subtable:

 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCjJU0lFyKkrMSwExPA2VYnUwBI2wCEKEjGBCRnAhY5gQiOFpDBE0gQmawNWhCHmaKMXGAgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Record ID" = _t, Brand = _t, #"Inv ID" = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{
        {"Record ID", type text}, {"Brand", type text}, {"Inv ID", type text}}),
    
    //Group by ID, then process each subtable
    #"Grouped Rows" = Table.Group(#"Changed Type", {"Record ID"}, {
        {"Rem Dups", (t)=>
            let 
                unique = Table.Distinct(t),
                #"Rem Nulls" = if Table.RowCount(unique) > 1
                    then Table.SelectRows(unique, each [Inv ID] <> null and [Inv ID] <> "") else unique
            in #"Rem Nulls",
                type table [Record ID=nullable text, Brand=nullable text, Inv ID=nullable text]}}),
    
    #"Expanded Rem Dups" = Table.ExpandTableColumn(#"Grouped Rows", "Rem Dups", {"Brand", "Inv ID"}, {"Brand", "Inv ID"})
in
    #"Expanded Rem Dups"

Data

ronrsnfld_0-1707340321890.png

Results

ronrsnfld_1-1707340359152.png

 

 

 

 

Helpful resources

Announcements
Power BI DataViz World Championships

Power BI Dataviz World Championships

The Power BI Data Visualization World Championships is back! Get ahead of the game and start preparing now!

December 2025 Power BI Update Carousel

Power BI Monthly Update - December 2025

Check out the December 2025 Power BI Holiday Recap!

FabCon Atlanta 2026 carousel

FabCon Atlanta 2026

Join us at FabCon Atlanta, March 16-20, for the ultimate Fabric, Power BI, AI and SQL community-led event. Save $200 with code FABCOMM.

Top Solution Authors