Forum Discussion

EaglesTony's avatar
EaglesTony
Post Prodigy
1 year ago
Solved

How can I group like rows ?

I have a table with the following:

 

NAME      KEY

Added      DATA-123

Removed  DATA-123

Committed DATA-456

Added       DATA-789

 

What I need is to know if the KEY has a Removed, to filter those out, thus the end I want is:

 

NAME            KEY

Committed    DATA-456

Added           DATA-789

 

I was thinking I could use a new table and merge some way with left joins, but is there a way to use the same table using some sort of group by function ?

 

 

Thanks

 

  • I decided on making a clone of the table and just having "removed" in there and merge it back and filter out those records with Key as null, it appears to work and needed to have this due to time sensitivity, but thanks all for your assitance and ideas.

27 Replies

  • Iwould not tackle this with group by.

    Add a custom colum:

    Table.AddColumn(Source, "To Be Removed", each Table.MatchesAnyRows(Source, (R) => R[KEY] = [KEY] and Text.Lower(R[NAME]) = "removed"))

    I am not at my laptop, so =have not tested this...

     

    Did I answer your question? Then please mark my post as the solution and make it easier to find for others having a similar problem.
    If I helped you, please click on the Thumbs Up to give Kudos.

     

    Kees Stolker

    A big fan of Power Query and Excel

  • Here is another possible solution...

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7DEFDQ2MsRmMlg4FgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Name = _t, Key = _t]),
        set_types = Table.TransformColumnTypes(Source,{{"Name", type text}, {"Key", type text}}),
        keys_to_remove = List.Transform(List.PositionOf(set_types[Name], "Removed", Occurrence.All), each set_types[Key]{_}),
        remove_keys = Table.SelectRows(set_types, each not List.Contains(keys_to_remove, [Key]))
    in
        remove_keys
    • EaglesTony's avatar
      EaglesTony
      Post Prodigy

      I've added this:

      = Table.AddColumn(#"Filtered Rows1", "Remove", each if Text.Contains([NAME], "-Removed", Comparer.OrdinalIgnoreCase) then "Y" else "N")

       

      However it gives me:

       

      NAME      KEY             REMOVE

      Added      DATA-123   N

      Removed  DATA-123   Y

      Committed DATA-456 N

      Added       DATA-789  N

       

      Now I have to figure out how to remove the 2 rows with DATA-123.

  • Greg_Deckler's avatar
    Greg_Deckler
    Community Champion

    EaglesTony Try this:

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7BUio0FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [NAME = _t, KEY = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"NAME", type text}, {"KEY", type text}}),
        #"Grouped Rows" = Table.Group(#"Changed Type", {"KEY"}, {{"Table", each _, type table [NAME=nullable text, KEY=nullable text]}}),
        #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each Table.ToList(Table.SelectColumns([Table],"NAME"))),
        #"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each List.Contains([Custom], "Removed")),
        #"Filtered Rows" = Table.SelectRows(#"Added Custom1", each ([Custom.1] = false)),
        #"Expanded Table" = Table.ExpandTableColumn(#"Filtered Rows", "Table", {"NAME"}, {"NAME"}),
        #"Removed Columns" = Table.RemoveColumns(#"Expanded Table",{"Custom", "Custom.1"})
    in
        #"Removed Columns"
  • And another approach, which seems to execute quite rapidly:

     

    Original

     

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7DEFDQ2MsRmMlg4FgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [NAME = _t, KEY = _t]),
        
        filter = Table.SelectRows(Source, (r)=>
            let 
                allKeys = Table.SelectRows(Source, each [KEY] = r[KEY])
            in 
                not List.Contains(allKeys[NAME],"Removed",Comparer.OrdinalIgnoreCase))
    in
        filter

     

     

    Result

     

     

    • EaglesTony's avatar
      EaglesTony
      Post Prodigy

      Can you explain from a manual part (i.e. selected Name column then group by) instead of the Advanced Editor View ?

      • ronrsnfld's avatar
        ronrsnfld
        Super User

        The code does not use GroupBy (and executes considerably faster). If you wanted to do it from the UI,

         1. when your table is showing, select the down arrow in the NAME column, and de-select "Removed"

          

         2. In the formula bar you will see something similar to below. "Source" will be the same as your  previous step.

               

         

         3. Replace what you see after the first comma with:

         

        (r)=>
                let 
                    allKeys = Table.SelectRows(Source, each [KEY] = r[KEY])
                in 
                    not List.Contains(allKeys[NAME],"Removed",Comparer.OrdinalIgnoreCase))

         

        resulting in:

         

         

         

         

         

         

    • EaglesTony's avatar
      EaglesTony
      Post Prodigy

      Not sure how to code this, as my last line is(I was trying to set some sort of flag and then filter out):

       

      #"Added Custom4" = Table.AddColumn(#"Filtered Rows1", "FeatureHasRemovalOnIt", each if Text.Contains([NAME], "-Removed", Comparer.OrdinalIgnoreCase) then "Y" else "N")

      • ronrsnfld's avatar
        ronrsnfld
        Super User

        With the code I provided there is no need for that step. The code will remove (filter out) the proper rows all in that single step. Perhaps if you pasted the original code into the Advanced Editor (replacing all that is there), you would understand better.

         

        Also, eliminating the "create a new column" step increases the efficiency/speed of the process.

         

        Perhaps:

        1. Duplicate your existing query
        2. Delete all the lines below where your full table appears.
        3. Then follow the instructions regarding inserting the line I Mentioned.
  •  

    Assuming the source:



    Power Query:

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7BUio0FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Name = _t, Key = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"Name", type text}, {"Key", type text}}),
        #"Trimmed Text" = Table.TransformColumns(#"Changed Type",{{"Name", Text.Trim, type text}, {"Key", Text.Trim, type text}}),
        #"Filtered Rows" = Table.SelectRows(#"Trimmed Text", each ([Name] = "Removed")),
        #"Merged Queries" = Table.NestedJoin(#"Trimmed Text", {"Key"}, #"Filtered Rows", {"Key"}, "Filtered Rows", JoinKind.LeftAnti),
        #"Removed Columns" = Table.RemoveColumns(#"Merged Queries",{"Filtered Rows"})
    in
        #"Removed Columns"

     

     

    or

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7BUio0FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Name = _t, Key = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"Name", type text}, {"Key", type text}}),
        #"Trimmed Text" = Table.TransformColumns(#"Changed Type",{{"Name", Text.Trim, type text}, {"Key", Text.Trim, type text}}),
        #"Added Custom" = Table.AddColumn(#"Trimmed Text", "To Be Removed", each Table.MatchesAnyRows(Source, (R) => R[Key] = [Key] and Text.Lower(R[Name]) = "removed")),
        #"Filtered Rows" = Table.SelectRows(#"Added Custom", each ([To Be Removed] = false)),
        #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"To Be Removed"})
    in
        #"Removed Columns"

     

    Output:

     

  • dufoq3's avatar
    dufoq3
    Community Champion

    Hi EaglesTony, you have many solutions here...
    Another one:

     

    Output:

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WckxJSU1R0lFycQxx1DU0MlaK1YlWCkrNzS/DFHbOz83NLClBSJiYmoElUAwxt7BUio0FAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [NAME = _t, KEY = _t]),
        FilteredName = Table.RemoveColumns(Table.Join(Source, "KEY", Table.RenameColumns(Table.SelectRows(Source, each [NAME] = "Removed")[[KEY]], {{"KEY", "KEY2"}}), "KEY2", JoinKind.LeftAnti), {"KEY2"})
    in
        FilteredName
  • I decided on making a clone of the table and just having "removed" in there and merge it back and filter out those records with Key as null, it appears to work and needed to have this due to time sensitivity, but thanks all for your assitance and ideas.