Don't miss your chance to take the Fabric Data Engineer (DP-600) exam for FREE! Find out how by attending the DP-600 session on April 23rd (pacific time), live or on-demand.
Learn moreNext up in the FabCon + SQLCon recap series: The roadmap for Microsoft SQL and Maximizing Developer experiences in Fabric. All sessions are available on-demand after the live show. Register now
Hi,
I have a dataset which have a null values and then lots of duplicates in the same column. I need a power query step which would first filter filter out rows with duplicate values not impacting the rows with null values in this column.
When I did these steps separately, I got below. Is there a way I can combine it into a single step?
Unselected null values - #"Filtered Rows" = Table.SelectRows(#"Added Custom", each ([Custom] <> null)),
Removed rows where this column has duplicate values - #"Removed Duplicates" = Table.Distinct(#"Filtered Rows", {"Custom"})
Thanks,
Muralidhar
Solved! Go to Solution.
Hi @murali5431 ,
Try the follow query:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlSK1YlWSsIgwUQyggkmUpDIVKXYWAA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
#"Replaced Value" = Table.ReplaceValue(Source,"",null,Replacer.ReplaceValue,{"Column1"}),
#"Removed Duplicates" = Table.FromRows(List.Distinct(Table.ToRows(#"Replaced Value"),each if _{0}=null then Text.NewGuid() else _{0}))
in
#"Removed Duplicates"
If the problem is still not resolved, please provide detailed error information or the expected result you expect. Let me know immediately, looking forward to your reply.
Best Regards,
Winniz
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
@amitchandak I did them as two steps as suggested:
First filtered out null values, the removed duplicates. Now when I want to remove the filter for null values, I dont find the null value filter itself after second step. Can you please assist?
Thanks,
Muralidhar
Hi @murali5431 ,
Try the follow query:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WSlSK1YlWSsIgwUQyggkmUpDIVKXYWAA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Column1 = _t]),
#"Replaced Value" = Table.ReplaceValue(Source,"",null,Replacer.ReplaceValue,{"Column1"}),
#"Removed Duplicates" = Table.FromRows(List.Distinct(Table.ToRows(#"Replaced Value"),each if _{0}=null then Text.NewGuid() else _{0}))
in
#"Removed Duplicates"
If the problem is still not resolved, please provide detailed error information or the expected result you expect. Let me know immediately, looking forward to your reply.
Best Regards,
Winniz
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
@amitchandak Thanks for the response! I would then need a third step to remove the filter for null values. Hence I felt it would be ideal to perform the duplicate removal as a single step.
Thanks again,
Muralidhar
If you have recently started exploring Fabric, we'd love to hear how it's going. Your feedback can help with product improvements.
A new Power BI DataViz World Championship is coming this June! Don't miss out on submitting your entry.
Share feedback directly with Fabric product managers, participate in targeted research studies and influence the Fabric roadmap.
| User | Count |
|---|---|
| 44 | |
| 43 | |
| 38 | |
| 19 | |
| 15 |
| User | Count |
|---|---|
| 68 | |
| 64 | |
| 31 | |
| 29 | |
| 24 |