Forum Discussion
Grouping Date sets within a month
- 5 years ago
Hello johnlhaase
check out this solution. I hope I got you right. The group-function with GroupKind.Local does the trick here
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("jdOxDYMwGEThXVwj2b7f4KTNAhkAsQL7l1iCQ1TJkyy78Nc93bqmmmtWUUlTij6P+9NbG88499c2/XGCLqBr0M3QLdB16F7QvaGrhUJQ5LufFEQxBV1MQRpTUMcUBDIFjUxBJlNQ6qICrUx5LfFa4rXEa4nXEq8luijRSYluKnineHTSL6rMpbAMLJ1zOwA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Start of Month" = _t, EmpID = _t, Fleet = _t, OnOff = _t, #"Rotation Date" = _t]), TransformDate = Table.TransformColumns(Source,{{"Rotation Date", each Date.From(_,"en-US"), type date},{"Start of Month", each Date.From(_,"en-US"), type date}}), Group = Table.Group(TransformDate, {"EmpID", "OnOff"}, {{"AllRows", each _, type table [Start of Month=date, EmpID=text, Fleet=text, OnOff=text, Rotation Date=date]}}, GroupKind.Local), AddMin = Table.AddColumn(Group, "Min", each List.Min([AllRows][Rotation Date]), type date), AddMax = Table.AddColumn(AddMin, "Max", each List.Max([AllRows][Rotation Date]), type date), ExpandGroupedTable = Table.ExpandTableColumn(AddMax, "AllRows", {"Start of Month", "Fleet", "Rotation Date"}, {"Start of Month", "Fleet", "Rotation Date"}) in ExpandGroupedTableCopy paste this code to the advanced editor in a new blank query to see how the solution works.
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy - 5 years ago
Hello
You are correct. I did ask about the same month. This is a large data set and when I was going through the restlts I realized some of the On/Off days transition into the following month.
The first solution is fantasitic and is very useful. I wish I realized the data set had the transition issue priro to my first post. The law of unintended consequences. Anyways I am trying to get the scripts to ignore the start of month all together and just get the min/max for each On/Off sets.
Thanks for your time
John
Hi johnlhaase - See if this works:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("jdOxDYMwGEThXVwj2b7f4KTNAhkAsQL7l1iCQ1TJkyy78Nc93bqmmmtWUUlTij6P+9NbG88499c2/XGCLqBr0M3QLdB16F7QvaGrhUJQ5LufFEQxBV1MQRpTUMcUBDIFjUxBJlNQ6qICrUx5LfFa4rXEa4nXEq8luijRSYluKnineHTSL6rMpbAMLJ1zOwA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [#"Start of Month" = _t, EmpID = _t, Fleet = _t, OnOff = _t, #"Rotation Date" = _t]),
#"Changed Type" = Table.TransformColumnTypes(Source,{{"Start of Month", type date}, {"Rotation Date", type date}}),
FakeNullRow =
{null} &
List.RemoveLastN(
Table.Column(
Source, "OnOff"
),
1
),
CombinedWithNullColumn = Table.ToColumns(#"Changed Type") & {FakeNullRow},
BackToTable = Table.FromColumns(CombinedWithNullColumn, Table.ColumnNames(#"Changed Type") & {"Previous OnOff Status"}),
#"Added Changed Status" = Table.AddColumn(BackToTable, "Changed Status", each if [Previous OnOff Status] = null then null else if [OnOff] = [Previous OnOff Status] then null else "Changed"),
#"Added Index" = Table.AddIndexColumn(#"Added Changed Status", "Index", 0, 1, Int64.Type),
#"Added Grouping" = Table.AddColumn(#"Added Index", "Grouping", each if [Changed Status] = "Changed" then [Index] else null),
#"Filled Down" = Table.FillDown(#"Added Grouping",{"Grouping"}),
#"Grouped Rows" = Table.Group(#"Filled Down", {"Grouping"}, {{"AllRows", each _, type table [Start of Month=nullable date, EmpID=nullable text, Fleet=nullable text, OnOff=nullable text, Rotation Date=nullable date, Previous OnOff Status=nullable text, Changed Status=nullable text, Index=number, Grouping=nullable number]}}),
#"Added Min Date" = Table.AddColumn(#"Grouped Rows", "Min Date", each Table.Min([AllRows], "Rotation Date")[Rotation Date]),
#"Added Max Date" = Table.AddColumn(#"Added Min Date", "Max Date", each Table.Max([AllRows], "Rotation Date")[Rotation Date]),
#"Expanded AllRows" = Table.ExpandTableColumn(#"Added Max Date", "AllRows", {"Start of Month", "EmpID", "Fleet", "OnOff", "Rotation Date"}, {"Start of Month", "EmpID", "Fleet", "OnOff", "Rotation Date"}),
#"Removed Other Columns" = Table.SelectColumns(#"Expanded AllRows",{"Start of Month", "EmpID", "Fleet", "OnOff", "Rotation Date", "Min Date", "Max Date"})
in
#"Removed Other Columns"
It turns this:
into this:
I did it by using some cool techniques from ImkeF at this blog article, but instead of using a custom function, I just did it in the code.
- It takes your OnOff column and adds it to the end of the current table as a new column, but shifted by one row.
- Adds a "Changed" text field to see if there is a change from on to blank (off I presume.)
- Adds an index. I just needed a sequence of numbers.
- If it is "Changed" give me the number in the Index column, otherwise, give me a null
- I then filled this column down. Now at every change, I have a different number.
- Grouped by that number, then used Table.Min() and Table.Max() to extract the min/max dates, then re-expanded your original columns.
Everything is UI driven (or in the Add Custom Column box) except for three steps:
- FakeNullRow
- CombinedWithNullColumn
- BackToTable
You can see what each step is doing though by looking at it as you walk through it.
How to use M code provided in a blank query:
1) In Power Query, select New Source, then Blank Query
2) On the Home ribbon, select "Advanced Editor" button
3) Remove everything you see, then paste the M code I've given you in that box.
4) Press Done
5) See this article if you need help using this M code in your model.
- johnlhaase5 years agoHelper I
Hello
Looks good. The only thing I cannot follow is the firat part with Compress and Deflate with all the letters in between. What does that do for the script?
Thanks again
John
- Jimmy8015 years agoCommunity Champion
Hello johnlhaase
this is to reproduce your dataset. You have to replace it with your datasource, or with the query already in place
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy- johnlhaase5 years agoHelper I
Thank you I will check it out!