Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get certified in Microsoft Fabric—for free! For a limited time, get a free DP-600 exam voucher to use by the end of 2024. Register now

Reply
Irwin
Helper IV
Helper IV

Filter column for unique values and another column in Power Query

Hi Everyone,

 

I have a very large table that I need to reduce in size. I need to filter column A to only have unique values and display a specific value Q, X or Z from column B. From column B I need to display Q>X>Z. That means if column B contains Q, then display Q, else X and else Z.

 

All help appreciated. 🙂


Example on data.

AB
1Z
1Z
1X
1Z
1Z
1Z
1Z
1Z
2Z
2Z
2Q
2Z
3Z
3Z
3Z
3Z
3Z
4Z
4Z
4Z
4Q
4Z

 

Desired output:

AB
1X
2Q
3Z
4Q
2 ACCEPTED SOLUTIONS
JorgePinho
Solution Sage
Solution Sage

Hello @Irwin !

You can sort in ascending order column B and then remove duplicates on column A.

View solution in original post

Hi,
It did not work. It seems this solution works in excel. However, PBI remembers the original sort and therefore cannot keep the value I want i column B once I remove duplicates.


I found a solution that work when applied to your suggestion. You need to use the table function called "Table.Buffer".

https://learn.microsoft.com/en-us/powerquery-m/table-buffer

A video can be seen here.
https://www.youtube.com/watch?v=rqDdnNxSgHQ&list=PLDz00l_jz6zzttb28XH8GHZNL6vvpBlkQ&index=23

View solution in original post

7 REPLIES 7
Anonymous
Not applicable

Hi @Irwin ,

Plaese refer to my steps: 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUYpSitVBZ0VgESOGZYSDFYgmZkwyy4QgKxBZLBYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [A = _t, B = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"A", Int64.Type}}),
    #"Grouped Rows" = Table.Group(#"Changed Type", {"A", "B"}, {{"Count", each Table.RowCount(_), Int64.Type}}),
    #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each List.Min(Table.SelectRows(#"Grouped Rows",(x)=>x[A]=[A])[Count])),
    #"Added Conditional Column" = Table.AddColumn(#"Added Custom", "Custom.1", each if [Count] = [Custom] then [B] else null),
    #"Filtered Rows" = Table.SelectRows(#"Added Conditional Column", each ([Custom.1] <> null)),
    #"Reordered Columns" = Table.ReorderColumns(#"Filtered Rows",{"A", "B", "Custom.1", "Count", "Custom"}),
    #"Removed Columns" = Table.RemoveColumns(#"Reordered Columns",{"Custom.1", "Count", "Custom"})
in
    #"Removed Columns"

vpollymsft_0-1668577401747.png

 

If it still does not help, please provide more details with your desired output.

 

Best Regards

Community Support Team _ Polly

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Hi,
Thank you for your kind help. This might be a solution, but I think it is not feasible to do on a data source with millions of rows.

JorgePinho
Solution Sage
Solution Sage

Hello @Irwin !

You can sort in ascending order column B and then remove duplicates on column A.

Wauv... this is a very simple solution and weirdly enough I think it might actually work. I will play around with it and see if there is no errors. 
Thank you.

I already used it sometimes and I think it doesn't make any errors. However, if it happens in your case please let me know 😁

Hi,
It did not work. It seems this solution works in excel. However, PBI remembers the original sort and therefore cannot keep the value I want i column B once I remove duplicates.


I found a solution that work when applied to your suggestion. You need to use the table function called "Table.Buffer".

https://learn.microsoft.com/en-us/powerquery-m/table-buffer

A video can be seen here.
https://www.youtube.com/watch?v=rqDdnNxSgHQ&list=PLDz00l_jz6zzttb28XH8GHZNL6vvpBlkQ&index=23

The first comment on that video link provided a very easy solution I think.

1) Sort as needed. (Sort Column B Ascending here)

2) Create Index column (*magic step*, will override original index during existence) 

3) Select Column A. Use Remove Duplicate rows. (will keep the first instance of each)

4) Remove index column. (if desired)

5) Sort by Column A or whatever you wish.

Phoen1x09_0-1679512406987.png

 

Helpful resources

Announcements
November Carousel

Fabric Community Update - November 2024

Find out what's new and trending in the Fabric Community.

Live Sessions with Fabric DB

Be one of the first to start using Fabric Databases

Starting December 3, join live sessions with database experts and the Fabric product team to learn just how easy it is to get started.

Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early Bird pricing ends December 9th.

Nov PBI Update Carousel

Power BI Monthly Update - November 2024

Check out the November 2024 Power BI update to learn about new features.