Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Don't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.

Reply
angle_Fbi
Helper II
Helper II

query to validate rows if dups

hi dears

i am facing an problem and i could not solve it in power query 

 

my issue is:  

 

i have a table which contain transaction ID and status. as you see the transaction id is duplicated ( which is fine for me ) but the status should be for each transaction id different.

 

so i need to keep the blue circles and remove the red circles.  do have any idea how to keep the blue cirles ( the different status ) ?

 

notice:  for status i only have 2 values ( G, F )

 

 

 

angle_Fbi_0-1721560941856.png

 

1 ACCEPTED SOLUTION
AmiraBedh
Most Valuable Professional
Most Valuable Professional

You select both columns > Remove Rows > Remove Duplicates :

AmiraBedh_0-1721566925731.pngAmiraBedh_1-1721566943879.png

Which is in Power Query : 

= Table.Distinct(#"yourPreviousStep")

Proud to be a Power BI Super User !

Microsoft Community : https://docs.microsoft.com/en-us/users/AmiraBedhiafi
Linkedin : https://www.linkedin.com/in/amira-bedhiafi/
StackOverflow : https://stackoverflow.com/users/9517769/amira-bedhiafi
C-Sharp Corner : https://www.c-sharpcorner.com/members/amira-bedhiafi
Power BI Community :https://community.powerbi.com/t5/user/viewprofilepage/user-id/332696

View solution in original post

5 REPLIES 5
AmiraBedh
Most Valuable Professional
Most Valuable Professional

You select both columns > Remove Rows > Remove Duplicates :

AmiraBedh_0-1721566925731.pngAmiraBedh_1-1721566943879.png

Which is in Power Query : 

= Table.Distinct(#"yourPreviousStep")

Proud to be a Power BI Super User !

Microsoft Community : https://docs.microsoft.com/en-us/users/AmiraBedhiafi
Linkedin : https://www.linkedin.com/in/amira-bedhiafi/
StackOverflow : https://stackoverflow.com/users/9517769/amira-bedhiafi
C-Sharp Corner : https://www.c-sharpcorner.com/members/amira-bedhiafi
Power BI Community :https://community.powerbi.com/t5/user/viewprofilepage/user-id/332696

god bless you it works

AmiraBedh
Most Valuable Professional
Most Valuable Professional

Glad to help 🙂


Proud to be a Power BI Super User !

Microsoft Community : https://docs.microsoft.com/en-us/users/AmiraBedhiafi
Linkedin : https://www.linkedin.com/in/amira-bedhiafi/
StackOverflow : https://stackoverflow.com/users/9517769/amira-bedhiafi
C-Sharp Corner : https://www.c-sharpcorner.com/members/amira-bedhiafi
Power BI Community :https://community.powerbi.com/t5/user/viewprofilepage/user-id/332696
jennratten
Super User
Super User

Hello  - Selecting both columns and selecting remove duplicates should produce the expected result.  It does look like your data needs to be cleaned up first though - you can see that some rows have an extra line and some do not.  Power Query is a case sensitive language that does not automatically omit whitespace and non-printable characters.  Without cleaning it first, you may end up with some records that appear to be the same but are not duplicates because they include some other differences that you cannot see. 

In this example below, I have cells in which the first two appear to be the same; the third appears to have the same value with an extra blank line. However, you can see that I have removed duplicates and still see three. This is because the first cell has a space at the end and the third has a new line character so the cells actually are different.

Here, I have added another column to convert the characters to their character numbers.  Now you can see that the first and last cells have characters that the second does not.

 

In your data the Transaction Id column appears to be formatted as a number so there shouldn't be any additional characters hiding in it.  The Status column, however, is text and could have additional characters.  You can add the column to see if there are any extra characters that need to be removed.  You could go ahead and trim and clean the column and that may solve the problem, but it also might not because that doesn't always catch everything, like if there are repeating spaces at the end.  Adding the column to check which characters are actually present is usually a good idea.

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45Wcnd0cjY0MlZQitWBc5RiYwE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column1", type text}}),
    #"Removed Duplicates" = Table.Distinct(#"Changed Type"),
    #"Added Custom" = Table.AddColumn(#"Removed Duplicates", "Custom", each Text.Combine ( List.Transform ( Text.ToList ( [Column1] ), each Number.ToText ( Character.ToNumber (_) ) ), "," ) )
in
    #"Added Custom"

 

jennratten_2-1721566425354.png

jennratten_3-1721566476975.png

@angle_Fbi

If this post helps to answer your questions, please consider marking it as a solution so others can find it more quickly when faced with a similar challenge.

Proud to be a Microsoft Fabric Super User

thank you very much for your support

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

December 2024

A Year in Review - December 2024

Find out what content was popular in the Fabric community during 2024.

Top Kudoed Authors