Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Next up in the FabCon + SQLCon recap series: The roadmap for Microsoft SQL and Maximizing Developer experiences in Fabric. All sessions are available on-demand after the live show. Register now

Reply
screamout_93
Frequent Visitor

Remove duplicate row for two multiple columns and condition on the third column

Hi guys, I need help on the power query . My intention here is to remove the duplicate value for line and date . what i want is to make it so the query will only remove rows that has null value in [From] if any duplicate are found. Table below as an example:-

 

before

DateLineFrom
1/1/20237011
1/1/2023701

Null

1/1/20237011
1/2/20237011

1/2/2023

7021

1/2/2023

702Null

1/3/2023

701Null

 

what i expected the result to be

1) Row from 1/1/2023 [Line] 701 with null value on [From] is removed.

2) Row from 1/2/203 [Line] 702 with null value on [From] is removed.

3) Row from 1/3/2023 [Line]701 with null value on [From] is kept because no duplicate valuefor [Date] and [Line]. 

DateLineFrom
1/1/20237011
1/1/20237011
1/2/20237011

1/2/2023

7021

1/3/2023

701Null

 

Appreciate if anyone can help me on this. Thank you!

1 ACCEPTED SOLUTION
AlienSx
Super User
Super User

Hello, @screamout_93 

let
    Source = your_table,
    #"Grouped Rows" = Table.Group(Source, {"Date", "Line"}, {{"Count", each Table.RowCount(_), Int64.Type}, {"rows", each _, type table [Date=datetime, Line=number, From=any]}}),
    #"Added Custom" = Table.AddColumn(#"Grouped Rows", "new_rows", each if [Count] > 1 then Table.SelectRows([rows], (x) => x[From] <> "Null") else [rows]),
    result = Table.Combine(#"Added Custom"[new_rows])
in
    result

View solution in original post

2 REPLIES 2
AlienSx
Super User
Super User

Hello, @screamout_93 

let
    Source = your_table,
    #"Grouped Rows" = Table.Group(Source, {"Date", "Line"}, {{"Count", each Table.RowCount(_), Int64.Type}, {"rows", each _, type table [Date=datetime, Line=number, From=any]}}),
    #"Added Custom" = Table.AddColumn(#"Grouped Rows", "new_rows", each if [Count] > 1 then Table.SelectRows([rows], (x) => x[From] <> "Null") else [rows]),
    result = Table.Combine(#"Added Custom"[new_rows])
in
    result

Thank you! your a lifesaver.

Helpful resources

Announcements
New to Fabric survey Carousel

New to Fabric Survey

If you have recently started exploring Fabric, we'd love to hear how it's going. Your feedback can help with product improvements.

Power BI DataViz World Championships carousel

Power BI DataViz World Championships - June 2026

A new Power BI DataViz World Championship is coming this June! Don't miss out on submitting your entry.

Join our Fabric User Panel

Join our Fabric User Panel

Share feedback directly with Fabric product managers, participate in targeted research studies and influence the Fabric roadmap.

March Power BI Update Carousel

Power BI Community Update - March 2026

Check out the March 2026 Power BI update to learn about new features.