Forum Discussion
How can I group like rows ?
I have a table with the following:
NAME KEY
Added DATA-123
Removed DATA-123
Committed DATA-456
Added DATA-789
What I need is to know if the KEY has a Removed, to filter those out, thus the end I want is:
NAME KEY
Committed DATA-456
Added DATA-789
I was thinking I could use a new table and merge some way with left joins, but is there a way to use the same table using some sort of group by function ?
Thanks
I decided on making a clone of the table and just having "removed" in there and merge it back and filter out those records with Key as null, it appears to work and needed to have this due to time sensitivity, but thanks all for your assitance and ideas.
27 Replies
- PwerQueryKeesSuper User
Iwould not tackle this with group by.
Add a custom colum:
Table.AddColumn(Source, "To Be Removed", each Table.MatchesAnyRows(Source, (R) => R[KEY] = [KEY] and Text.Lower(R[NAME]) = "removed"))I am not at my laptop, so =have not tested this...
Did I answer your question? Then please mark my post as the solution and make it easier to find for others having a similar problem.
If I helped you, please click on the Thumbs Up to give Kudos.Kees Stolker
A big fan of Power Query and Excel
- jgeddesSuper User
Here is another possible solution...
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7DEFDQ2MsRmMlg4FgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Name = _t, Key = _t]), set_types = Table.TransformColumnTypes(Source,{{"Name", type text}, {"Key", type text}}), keys_to_remove = List.Transform(List.PositionOf(set_types[Name], "Removed", Occurrence.All), each set_types[Key]{_}), remove_keys = Table.SelectRows(set_types, each not List.Contains(keys_to_remove, [Key])) in remove_keys- EaglesTonyPost Prodigy
I've added this:
= Table.AddColumn(#"Filtered Rows1", "Remove", each if Text.Contains([NAME], "-Removed", Comparer.OrdinalIgnoreCase) then "Y" else "N")
However it gives me:
NAME KEY REMOVE
Added DATA-123 N
Removed DATA-123 Y
Committed DATA-456 N
Added DATA-789 N
Now I have to figure out how to remove the 2 rows with DATA-123.
- Greg_DecklerCommunity Champion
EaglesTony I have attached the PBIX file that I used. You want the Table query.
- Greg_DecklerCommunity Champion
EaglesTony Try this:
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7BUio0FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [NAME = _t, KEY = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"NAME", type text}, {"KEY", type text}}), #"Grouped Rows" = Table.Group(#"Changed Type", {"KEY"}, {{"Table", each _, type table [NAME=nullable text, KEY=nullable text]}}), #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each Table.ToList(Table.SelectColumns([Table],"NAME"))), #"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each List.Contains([Custom], "Removed")), #"Filtered Rows" = Table.SelectRows(#"Added Custom1", each ([Custom.1] = false)), #"Expanded Table" = Table.ExpandTableColumn(#"Filtered Rows", "Table", {"NAME"}, {"NAME"}), #"Removed Columns" = Table.RemoveColumns(#"Expanded Table",{"Custom", "Custom.1"}) in #"Removed Columns" - ronrsnfldSuper User
And another approach, which seems to execute quite rapidly:
Original
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7DEFDQ2MsRmMlg4FgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [NAME = _t, KEY = _t]), filter = Table.SelectRows(Source, (r)=> let allKeys = Table.SelectRows(Source, each [KEY] = r[KEY]) in not List.Contains(allKeys[NAME],"Removed",Comparer.OrdinalIgnoreCase)) in filterResult
- EaglesTonyPost Prodigy
Can you explain from a manual part (i.e. selected Name column then group by) instead of the Advanced Editor View ?
- ronrsnfldSuper User
The code does not use GroupBy (and executes considerably faster). If you wanted to do it from the UI,
1. when your table is showing, select the down arrow in the NAME column, and de-select "Removed"
2. In the formula bar you will see something similar to below. "Source" will be the same as your previous step.
3. Replace what you see after the first comma with:
(r)=> let allKeys = Table.SelectRows(Source, each [KEY] = r[KEY]) in not List.Contains(allKeys[NAME],"Removed",Comparer.OrdinalIgnoreCase))resulting in:
- EaglesTonyPost Prodigy
Not sure how to code this, as my last line is(I was trying to set some sort of flag and then filter out):
#"Added Custom4" = Table.AddColumn(#"Filtered Rows1", "FeatureHasRemovalOnIt", each if Text.Contains([NAME], "-Removed", Comparer.OrdinalIgnoreCase) then "Y" else "N")
- ronrsnfldSuper User
With the code I provided there is no need for that step. The code will remove (filter out) the proper rows all in that single step. Perhaps if you pasted the original code into the Advanced Editor (replacing all that is there), you would understand better.
Also, eliminating the "create a new column" step increases the efficiency/speed of the process.
Perhaps:
- Duplicate your existing query
- Delete all the lines below where your full table appears.
- Then follow the instructions regarding inserting the line I Mentioned.
- sevenhillsSuper User
Assuming the source:
Power Query:let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7BUio0FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Name = _t, Key = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Name", type text}, {"Key", type text}}), #"Trimmed Text" = Table.TransformColumns(#"Changed Type",{{"Name", Text.Trim, type text}, {"Key", Text.Trim, type text}}), #"Filtered Rows" = Table.SelectRows(#"Trimmed Text", each ([Name] = "Removed")), #"Merged Queries" = Table.NestedJoin(#"Trimmed Text", {"Key"}, #"Filtered Rows", {"Key"}, "Filtered Rows", JoinKind.LeftAnti), #"Removed Columns" = Table.RemoveColumns(#"Merged Queries",{"Filtered Rows"}) in #"Removed Columns"or
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7BUio0FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Name = _t, Key = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Name", type text}, {"Key", type text}}), #"Trimmed Text" = Table.TransformColumns(#"Changed Type",{{"Name", Text.Trim, type text}, {"Key", Text.Trim, type text}}), #"Added Custom" = Table.AddColumn(#"Trimmed Text", "To Be Removed", each Table.MatchesAnyRows(Source, (R) => R[Key] = [Key] and Text.Lower(R[Name]) = "removed")), #"Filtered Rows" = Table.SelectRows(#"Added Custom", each ([To Be Removed] = false)), #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"To Be Removed"}) in #"Removed Columns"Output:
- dufoq3Community Champion
Hi EaglesTony, you have many solutions here...
Another one:Output:
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7BUio0FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [NAME = _t, KEY = _t]), FilteredName = Table.RemoveColumns(Table.Join(Source, "KEY", Table.RenameColumns(Table.SelectRows(Source, each [NAME] = "Removed")[[KEY]], {{"KEY", "KEY2"}}), "KEY2", JoinKind.LeftAnti), {"KEY2"}) in FilteredName - EaglesTonyPost Prodigy
I decided on making a clone of the table and just having "removed" in there and merge it back and filter out those records with Key as null, it appears to work and needed to have this due to time sensitivity, but thanks all for your assitance and ideas.