Starting December 3, join live sessions with database experts and the Microsoft product team to learn just how easy it is to get started
Learn moreGet certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now
Hi all,
I have a employee history table. I noticed some duplicated rows. How can I remove the duplicated row, based on the condition:
1. The same Employee Number
2. Chg Rsn="901"? (901 means new hired employee, new hire must be unique)
Thank you in advance!
Bei
Solved! Go to Solution.
Hi,
Here is one way to do this:
Example (we will remove one of the rows in yellow):
Here is the PQ used:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMjQyVtJRsjQwVIrVweSZgHlGWHmmWHhAfbEA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Empid = _t, code = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Empid", Int64.Type}, {"code", Int64.Type}}),
#"Filtered Rows1" = Table.SelectRows(#"Changed Type", each [code] <> 901), //this table contains non 901 rows
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [code] = 901),
#"Removed Duplicates" = Table.Distinct(#"Filtered Rows", {"Empid"}), //this table contains non unique 901 rows
#"Appended Query" = Table.Combine({#"Removed Duplicates", #"Filtered Rows1"}) //here we combine the two to get the desired result
in
#"Appended Query"
End result:
As we can see the non-desired row is now removed.
I hope this post helps to solve your issue and if it does consider accepting it as a solution and giving the post a thumbs up!
My LinkedIn: https://www.linkedin.com/in/n%C3%A4ttiahov-00001/
Proud to be a Super User!
Hi,
Here is one way to do this:
Example (we will remove one of the rows in yellow):
Here is the PQ used:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMjQyVtJRsjQwVIrVweSZgHlGWHmmWHhAfbEA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Empid = _t, code = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Empid", Int64.Type}, {"code", Int64.Type}}),
#"Filtered Rows1" = Table.SelectRows(#"Changed Type", each [code] <> 901), //this table contains non 901 rows
#"Filtered Rows" = Table.SelectRows(#"Changed Type", each [code] = 901),
#"Removed Duplicates" = Table.Distinct(#"Filtered Rows", {"Empid"}), //this table contains non unique 901 rows
#"Appended Query" = Table.Combine({#"Removed Duplicates", #"Filtered Rows1"}) //here we combine the two to get the desired result
in
#"Appended Query"
End result:
As we can see the non-desired row is now removed.
I hope this post helps to solve your issue and if it does consider accepting it as a solution and giving the post a thumbs up!
My LinkedIn: https://www.linkedin.com/in/n%C3%A4ttiahov-00001/
Proud to be a Super User!
This solution is genius! How did you come up with it?
Hi,
@Anonymous
I created a flowchart with the categories required. In the end the problem is that we are trying to eliminate rows with certain conditions. So reversely, if we include all but the rows we want to eliminate we get the desired outcome.
Proud to be a Super User!
Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.
March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early Bird pricing ends December 9th.
User | Count |
---|---|
22 | |
21 | |
17 | |
14 | |
11 |
User | Count |
---|---|
43 | |
35 | |
25 | |
24 | |
22 |