Forum Discussion
Transforming schedule data
- 3 years ago
Ok, that was a pretty simple change. Rather than adding 6 days, I just went from start to end.
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUbLUN9Q3MjAyAjONYUwDJGyiZwokDeEisTrRShDlhgYw9YZG+sZwcwyxYIRWY4hWI4StRnBrjcBWGWNhQfXHAgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, Start_Date = _t, End_Date = _t, Monday_Hours = _t, Tuesday_Hours = _t, Wednesday_Hours = _t, Thursday_Hours = _t, Friday_Hours = _t, Saturday_Hours = _t, Sunday_Hours = _t]), #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Source, {"ID", "Start_Date", "End_Date"}, "Attribute", "Value"), #"Changed Type" = Table.TransformColumnTypes(#"Unpivoted Other Columns",{{"Start_Date", type date}, {"End_Date", type date}, {"Value", Currency.Type}}), #"Extracted Text Before Delimiter" = Table.TransformColumns(#"Changed Type", {{"Attribute", each Text.BeforeDelimiter(_, "_"), type text}}), #"Filtered out Zeros" = Table.SelectRows(#"Extracted Text Before Delimiter", each ([Value] <> 0)), #"Duplicated Column" = Table.DuplicateColumn(#"Filtered out Zeros", "Start_Date", "Date Bracket"), AddDateRange = Table.AddColumn( #"Duplicated Column", "Date Range", each let varStartDate = Number.From([Start_Date]), varEndDate = Number.From([End_Date]) in {varStartDate..varEndDate} ), #"Expanded Date Range" = Table.ExpandListColumn(AddDateRange, "Date Range"), #"Changed Type1" = Table.TransformColumnTypes(#"Expanded Date Range",{{"Date Range", type date}}), #"Inserted Day Name" = Table.AddColumn(#"Changed Type1", "Day Name", each Date.DayOfWeekName([Date Range]), type text), #"Added Match" = Table.AddColumn(#"Inserted Day Name", "Match", each [Attribute] = [Day Name], Logical.Type), #"Filtered for Match is TRUE" = Table.SelectRows(#"Added Match", each ([Match] = true)), #"Removed Other Columns" = Table.SelectColumns(#"Filtered for Match is TRUE",{"ID", "Date Range", "Value"}), #"Grouped Rows" = Table.Group(#"Removed Other Columns", {"ID", "Date Range"}, {{"Hours", each List.Sum([Value]), type nullable number}}) in #"Grouped Rows"So just for ID 3, I get this. Originally 3 was 5 rows, but spanned 2 weeks, so it makes sense to me it now has 10 rows. As you said, ID 2 is dozens of rows...
rbreneman here is what I came up with:
Here is my code:
let
Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUbLUN9Q3MjAyAjONYUwDJGyiZwokDeEisTrRShDlhgYw9YZG+sZwcwyxYIRWY4hWI4StRnBrjcBWGWNhQfXHAgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, Start_Date = _t, End_Date = _t, Monday_Hours = _t, Tuesday_Hours = _t, Wednesday_Hours = _t, Thursday_Hours = _t, Friday_Hours = _t, Saturday_Hours = _t, Sunday_Hours = _t]),
#"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Source, {"ID", "Start_Date", "End_Date"}, "Attribute", "Value"),
#"Changed Type" = Table.TransformColumnTypes(#"Unpivoted Other Columns",{{"Start_Date", type date}, {"End_Date", type date}, {"Value", Currency.Type}}),
#"Extracted Text Before Delimiter" = Table.TransformColumns(#"Changed Type", {{"Attribute", each Text.BeforeDelimiter(_, "_"), type text}}),
#"Filtered out Zeros" = Table.SelectRows(#"Extracted Text Before Delimiter", each ([Value] <> 0)),
#"Duplicated Column" = Table.DuplicateColumn(#"Filtered out Zeros", "Start_Date", "Date Bracket"),
AddDateRange =
Table.AddColumn(
#"Duplicated Column",
"Date Range",
each
let
varStartDate = Number.From([Start_Date])
in
{varStartDate..varStartDate+6}
),
#"Expanded Date Range" = Table.ExpandListColumn(AddDateRange, "Date Range"),
#"Changed Type1" = Table.TransformColumnTypes(#"Expanded Date Range",{{"Date Range", type date}}),
#"Inserted Day Name" = Table.AddColumn(#"Changed Type1", "Day Name", each Date.DayOfWeekName([Date Range]), type text),
#"Added Match" = Table.AddColumn(#"Inserted Day Name", "Match", each [Attribute] = [Day Name], Logical.Type),
#"Filtered for Match is TRUE" = Table.SelectRows(#"Added Match", each ([Match] = true)),
#"Removed Other Columns" = Table.SelectColumns(#"Filtered for Match is TRUE",{"ID", "Date Range", "Value"})
in
#"Removed Other Columns"
Here is what I did:
- Unpivoted all but the ID, StartDate and End Date. That gives me a clean table.
- The Attribute column now has your "Monday_Hours" field name in it. I got rid of the _Hours part.
- Filtered out 0 rows for the value.
- Now I added a date range based on the Start Date Plus 6 days, so a full week. This is a nested list in the "AddDateRange" step.
- Expanded that and changed the Date Range to a date data type.
- I inserted the name of the day based on that date range.
- I then compared that day name to your Attribute (your old column) and only kept those that matched.
- Removed all of the unnecessary columns.
If this isn't what you need, please be more explicit in how you expect to arrive at the results. Your example wasn't 100% clear to me, but my solution makes sense. It may not be right for your needs though.
How to use M code provided in a blank query:
1) In Power Query, select New Source, then Blank Query
2) On the Home ribbon, select "Advanced Editor" button
3) Remove everything you see, then paste the M code I've given you in that box.
4) Press Done
5) See this article if you need help using this M code in your model.
- rbreneman3 years agoHelper II
Hi edhans ,
Thanks for your reply. That gets me close, however for the rows in the original table that span multiple weeks I need to be able to reflect all those dates in the final solution. For example ID 003 in the original table has a start date of 9/12/2022 and an end date of 9/23/2022. This spans two weeks, so the pivot would need to result in 12 dates (excluding zeros). Essentially in that example, there would be a 9/12/2022 of 2.5 hours and also a 9/19/2022 of 2.5 hours. ID 002 is a much larger range going all the way to the end of the year, so it would encompass all dates in that range. Basically the hours for each weekday would need a coresponding row for every occurance of that weekday during the date range.
Hopefully that makes sense and adds neccessary context?
Thanks so much for your help!
- edhans3 years agoCommunity Champion
Ok, that was a pretty simple change. Rather than adding 6 days, I just went from start to end.
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUbLUN9Q3MjAyAjONYUwDJGyiZwokDeEisTrRShDlhgYw9YZG+sZwcwyxYIRWY4hWI4StRnBrjcBWGWNhQfXHAgA=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [ID = _t, Start_Date = _t, End_Date = _t, Monday_Hours = _t, Tuesday_Hours = _t, Wednesday_Hours = _t, Thursday_Hours = _t, Friday_Hours = _t, Saturday_Hours = _t, Sunday_Hours = _t]), #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Source, {"ID", "Start_Date", "End_Date"}, "Attribute", "Value"), #"Changed Type" = Table.TransformColumnTypes(#"Unpivoted Other Columns",{{"Start_Date", type date}, {"End_Date", type date}, {"Value", Currency.Type}}), #"Extracted Text Before Delimiter" = Table.TransformColumns(#"Changed Type", {{"Attribute", each Text.BeforeDelimiter(_, "_"), type text}}), #"Filtered out Zeros" = Table.SelectRows(#"Extracted Text Before Delimiter", each ([Value] <> 0)), #"Duplicated Column" = Table.DuplicateColumn(#"Filtered out Zeros", "Start_Date", "Date Bracket"), AddDateRange = Table.AddColumn( #"Duplicated Column", "Date Range", each let varStartDate = Number.From([Start_Date]), varEndDate = Number.From([End_Date]) in {varStartDate..varEndDate} ), #"Expanded Date Range" = Table.ExpandListColumn(AddDateRange, "Date Range"), #"Changed Type1" = Table.TransformColumnTypes(#"Expanded Date Range",{{"Date Range", type date}}), #"Inserted Day Name" = Table.AddColumn(#"Changed Type1", "Day Name", each Date.DayOfWeekName([Date Range]), type text), #"Added Match" = Table.AddColumn(#"Inserted Day Name", "Match", each [Attribute] = [Day Name], Logical.Type), #"Filtered for Match is TRUE" = Table.SelectRows(#"Added Match", each ([Match] = true)), #"Removed Other Columns" = Table.SelectColumns(#"Filtered for Match is TRUE",{"ID", "Date Range", "Value"}), #"Grouped Rows" = Table.Group(#"Removed Other Columns", {"ID", "Date Range"}, {{"Hours", each List.Sum([Value]), type nullable number}}) in #"Grouped Rows"So just for ID 3, I get this. Originally 3 was 5 rows, but spanned 2 weeks, so it makes sense to me it now has 10 rows. As you said, ID 2 is dozens of rows...
- rbreneman3 years agoHelper II
edhans thanks so much! This is great! I did some quick validation on my dataset and everything looks good. I greatly appreciate your time and explaining the steps you took to get to the solution.
AlexisOlson Thanks for your insight as well. My dataset is small enough right now that the run time is not of concern, but this is good to know so I can build my report to scale.