Starting December 3, join live sessions with database experts and the Microsoft product team to learn just how easy it is to get started
Learn moreGet certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now
Hi Everyone,
I have a very large table that I need to reduce in size. I need to filter column A to only have unique values and display a specific value Q, X or Z from column B. From column B I need to display Q>X>Z. That means if column B contains Q, then display Q, else X and else Z.
All help appreciated. 🙂
Example on data.
A | B |
1 | Z |
1 | Z |
1 | X |
1 | Z |
1 | Z |
1 | Z |
1 | Z |
1 | Z |
2 | Z |
2 | Z |
2 | Q |
2 | Z |
3 | Z |
3 | Z |
3 | Z |
3 | Z |
3 | Z |
4 | Z |
4 | Z |
4 | Z |
4 | Q |
4 | Z |
Desired output:
A | B |
1 | X |
2 | Q |
3 | Z |
4 | Q |
Solved! Go to Solution.
Hello @Irwin !
You can sort in ascending order column B and then remove duplicates on column A.
Hi,
It did not work. It seems this solution works in excel. However, PBI remembers the original sort and therefore cannot keep the value I want i column B once I remove duplicates.
I found a solution that work when applied to your suggestion. You need to use the table function called "Table.Buffer".
https://learn.microsoft.com/en-us/powerquery-m/table-buffer
A video can be seen here.
https://www.youtube.com/watch?v=rqDdnNxSgHQ&list=PLDz00l_jz6zzttb28XH8GHZNL6vvpBlkQ&index=23
Hi @Irwin ,
Plaese refer to my steps:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUYpSitVBZ0VgESOGZYSDFYgmZkwyy4QgKxBZLBYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [A = _t, B = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"A", Int64.Type}}),
#"Grouped Rows" = Table.Group(#"Changed Type", {"A", "B"}, {{"Count", each Table.RowCount(_), Int64.Type}}),
#"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each List.Min(Table.SelectRows(#"Grouped Rows",(x)=>x[A]=[A])[Count])),
#"Added Conditional Column" = Table.AddColumn(#"Added Custom", "Custom.1", each if [Count] = [Custom] then [B] else null),
#"Filtered Rows" = Table.SelectRows(#"Added Conditional Column", each ([Custom.1] <> null)),
#"Reordered Columns" = Table.ReorderColumns(#"Filtered Rows",{"A", "B", "Custom.1", "Count", "Custom"}),
#"Removed Columns" = Table.RemoveColumns(#"Reordered Columns",{"Custom.1", "Count", "Custom"})
in
#"Removed Columns"
If it still does not help, please provide more details with your desired output.
Best Regards
Community Support Team _ Polly
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hi,
Thank you for your kind help. This might be a solution, but I think it is not feasible to do on a data source with millions of rows.
Hello @Irwin !
You can sort in ascending order column B and then remove duplicates on column A.
Wauv... this is a very simple solution and weirdly enough I think it might actually work. I will play around with it and see if there is no errors.
Thank you.
I already used it sometimes and I think it doesn't make any errors. However, if it happens in your case please let me know 😁
Hi,
It did not work. It seems this solution works in excel. However, PBI remembers the original sort and therefore cannot keep the value I want i column B once I remove duplicates.
I found a solution that work when applied to your suggestion. You need to use the table function called "Table.Buffer".
https://learn.microsoft.com/en-us/powerquery-m/table-buffer
A video can be seen here.
https://www.youtube.com/watch?v=rqDdnNxSgHQ&list=PLDz00l_jz6zzttb28XH8GHZNL6vvpBlkQ&index=23
The first comment on that video link provided a very easy solution I think.
1) Sort as needed. (Sort Column B Ascending here)
2) Create Index column (*magic step*, will override original index during existence)
3) Select Column A. Use Remove Duplicate rows. (will keep the first instance of each)
4) Remove index column. (if desired)
5) Sort by Column A or whatever you wish.
Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.
March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early Bird pricing ends December 9th.
User | Count |
---|---|
94 | |
85 | |
82 | |
69 | |
49 |
User | Count |
---|---|
143 | |
123 | |
107 | |
61 | |
55 |