Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
5 years ago
Solved

Last 100 rows for each Unique value in Column

The SQL table I'm pulling has millions of rows and I only need the last 100 records for each unique program in a column. See generic example below for pulling last 3 records for each unique program i...
Need Help
  • mahoneypat's avatar
    mahoneypat
    5 years ago

    Here is one way to do it by grouping your data, sorting and keep the last 3 dates for each.  To see how it works, just create a blank query, go to Advanced Editor, and replace the text there with the M code below that transforms your example data.

     

    let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("Zc/JCcAwDETRXnQ2RFK8HrN0Ydx/GyFghzBzffwRdu9im22urhLklhFe8AXHhB2LiEVacE7ICAUnFY82nJhiYkaNU/O99loSaZWoySSFpJI0/JXrT8YD", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Program = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}}),
    #"Grouped Rows" = Table.Group(#"Changed Type", {"Program"}, {{"AllRows", each _, type table [Date=nullable text, Program=nullable text]}}),
    #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Last3Dates", each Table.FirstN(Table.Sort([AllRows], {{"Date", Order.Descending}}),3)),
    #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"AllRows"}),
    #"Expanded Last3Dates" = Table.ExpandTableColumn(#"Removed Columns", "Last3Dates", {"Date"}, {"Date"})
in
    #"Expanded Last3Dates"

     

    Regards,

    Pat

     

mahoneypat's avatar
mahoneypat
Microsoft Employee
5 years ago

Here is one way to do it by grouping your data, sorting and keep the last 3 dates for each.  To see how it works, just create a blank query, go to Advanced Editor, and replace the text there with the M code below that transforms your example data.

 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("Zc/JCcAwDETRXnQ2RFK8HrN0Ydx/GyFghzBzffwRdu9im22urhLklhFe8AXHhB2LiEVacE7ICAUnFY82nJhiYkaNU/O99loSaZWoySSFpJI0/JXrT8YD", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Date = _t, Program = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Date", type date}}),
    #"Grouped Rows" = Table.Group(#"Changed Type", {"Program"}, {{"AllRows", each _, type table [Date=nullable text, Program=nullable text]}}),
    #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Last3Dates", each Table.FirstN(Table.Sort([AllRows], {{"Date", Order.Descending}}),3)),
    #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"AllRows"}),
    #"Expanded Last3Dates" = Table.ExpandTableColumn(#"Removed Columns", "Last3Dates", {"Date"}, {"Date"})
in
    #"Expanded Last3Dates"

 

Regards,

Pat