Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
Irwin
Helper IV
Helper IV

Filter column for unique values and another column in Power Query

Hi Everyone,

 

I have a very large table that I need to reduce in size. I need to filter column A to only have unique values and display a specific value Q, X or Z from column B. From column B I need to display Q>X>Z. That means if column B contains Q, then display Q, else X and else Z.

 

All help appreciated. 🙂


Example on data.

AB
1Z
1Z
1X
1Z
1Z
1Z
1Z
1Z
2Z
2Z
2Q
2Z
3Z
3Z
3Z
3Z
3Z
4Z
4Z
4Z
4Q
4Z

 

Desired output:

AB
1X
2Q
3Z
4Q
2 ACCEPTED SOLUTIONS
JorgePinho
Solution Sage
Solution Sage

Hello @Irwin !

You can sort in ascending order column B and then remove duplicates on column A.

View solution in original post

Hi,
It did not work. It seems this solution works in excel. However, PBI remembers the original sort and therefore cannot keep the value I want i column B once I remove duplicates.


I found a solution that work when applied to your suggestion. You need to use the table function called "Table.Buffer".

https://learn.microsoft.com/en-us/powerquery-m/table-buffer

A video can be seen here.
https://www.youtube.com/watch?v=rqDdnNxSgHQ&list=PLDz00l_jz6zzttb28XH8GHZNL6vvpBlkQ&index=23

View solution in original post

7 REPLIES 7
v-rongtiep-msft
Community Support
Community Support

Hi @Irwin ,

Plaese refer to my steps: 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUYpSitVBZ0VgESOGZYSDFYgmZkwyy4QgKxBZLBYA", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [A = _t, B = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"A", Int64.Type}}),
    #"Grouped Rows" = Table.Group(#"Changed Type", {"A", "B"}, {{"Count", each Table.RowCount(_), Int64.Type}}),
    #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each List.Min(Table.SelectRows(#"Grouped Rows",(x)=>x[A]=[A])[Count])),
    #"Added Conditional Column" = Table.AddColumn(#"Added Custom", "Custom.1", each if [Count] = [Custom] then [B] else null),
    #"Filtered Rows" = Table.SelectRows(#"Added Conditional Column", each ([Custom.1] <> null)),
    #"Reordered Columns" = Table.ReorderColumns(#"Filtered Rows",{"A", "B", "Custom.1", "Count", "Custom"}),
    #"Removed Columns" = Table.RemoveColumns(#"Reordered Columns",{"Custom.1", "Count", "Custom"})
in
    #"Removed Columns"

vpollymsft_0-1668577401747.png

 

If it still does not help, please provide more details with your desired output.

 

Best Regards

Community Support Team _ Polly

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Hi,
Thank you for your kind help. This might be a solution, but I think it is not feasible to do on a data source with millions of rows.

JorgePinho
Solution Sage
Solution Sage

Hello @Irwin !

You can sort in ascending order column B and then remove duplicates on column A.

Wauv... this is a very simple solution and weirdly enough I think it might actually work. I will play around with it and see if there is no errors. 
Thank you.

I already used it sometimes and I think it doesn't make any errors. However, if it happens in your case please let me know 😁

Hi,
It did not work. It seems this solution works in excel. However, PBI remembers the original sort and therefore cannot keep the value I want i column B once I remove duplicates.


I found a solution that work when applied to your suggestion. You need to use the table function called "Table.Buffer".

https://learn.microsoft.com/en-us/powerquery-m/table-buffer

A video can be seen here.
https://www.youtube.com/watch?v=rqDdnNxSgHQ&list=PLDz00l_jz6zzttb28XH8GHZNL6vvpBlkQ&index=23

The first comment on that video link provided a very easy solution I think.

1) Sort as needed. (Sort Column B Ascending here)

2) Create Index column (*magic step*, will override original index during existence) 

3) Select Column A. Use Remove Duplicate rows. (will keep the first instance of each)

4) Remove index column. (if desired)

5) Sort by Column A or whatever you wish.

Phoen1x09_0-1679512406987.png

 

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

Power BI Carousel June 2024

Power BI Monthly Update - June 2024

Check out the June 2024 Power BI update to learn about new features.

RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.