Forum Discussion
Calculate Difference previous date in Power Query ( M ) (only)
- 9 years ago
Find below 2 queries.
The first query is the proposed solution.
This includes a function which you can't step through in the query editor.
The second query is an amended vesion for 1 product, with some lines commented out and some other lines added, so you can see the steps that are part of the function in the first query.
FYI: I first created the query for 1 product and turned this into a function for the final solution.
I tested with about 25,000 rows of data in Excel and it runs for about 2 seconds (!).
Proposed solution:
let Source = TestData, #"Grouped Rows" = Table.Group(Source, {"Product"}, {{"AllData", each fnPriceDifference(_), type table}}), fnPriceDifference = (MyTable as table) as table => let #"Sorted Rows" = Table.Buffer(Table.Sort(MyTable,{{"date", Order.Descending}})), #"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1), #"Added Index1" = Table.AddIndexColumn(#"Added Index", "Index.1", 1, 1), #"Merged Queries" = Table.NestedJoin(#"Added Index1",{"Index.1"},#"Added Index1",{"Index"},"PreviousDate",JoinKind.LeftOuter), #"Expanded PreviousDate" = Table.ExpandTableColumn(#"Merged Queries", "PreviousDate", {"Cost"}, {"PreviousDate.Cost"}), #"Added Custom" = Table.AddColumn(#"Expanded PreviousDate", "Difference", each if [PreviousDate.Cost] = null then null else [Cost] - [PreviousDate.Cost]), #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index", "Index.1", "PreviousDate.Cost"}) in #"Removed Columns", #"Expanded AllData" = Table.ExpandTableColumn(#"Grouped Rows", "AllData", {"date", "Campaign", "Cost", "id", "Difference"}, {"date", "Campaign", "Cost", "id", "Difference"}) in #"Expanded AllData"Code for 1 product only (just to see all steps):
let Source = TestData, // #"Grouped Rows" = Table.Group(Source, {"Product"}, {{"AllData", each fnPriceDifference(_), type table}}), #"Grouped Rows" = Table.Group(Source, {"Product"}, {{"AllData", each _, type table}}), MyTable = #"Grouped Rows"{[Product="A"]}[AllData], // fnPriceDifference = (MyTable as table) as table => // let #"Sorted Rows" = Table.Buffer(Table.Sort(MyTable,{{"date", Order.Descending}})), #"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1), #"Added Index1" = Table.AddIndexColumn(#"Added Index", "Index.1", 1, 1), #"Merged Queries" = Table.NestedJoin(#"Added Index1",{"Index.1"},#"Added Index1",{"Index"},"PreviousDate",JoinKind.LeftOuter), #"Expanded PreviousDate" = Table.ExpandTableColumn(#"Merged Queries", "PreviousDate", {"Cost"}, {"PreviousDate.Cost"}), #"Added Custom" = Table.AddColumn(#"Expanded PreviousDate", "Difference", each if [PreviousDate.Cost] = null then null else [Cost] - [PreviousDate.Cost]), #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index", "Index.1", "PreviousDate.Cost"}) // in // #"Removed Columns", // #"Expanded AllData" = Table.ExpandTableColumn(#"Grouped Rows", "AllData", {"date", "Campaign", "Cost", "id", "Difference"}, {"date", "Campaign", "Cost", "id", "Difference"}) in // #"Expanded AllData" #"Removed Columns"
Hi MarcelBeug,
regarding the first question, the products have only one cost by date. The second is a bit more complicated, for the first date, there is no expected value because you can't compare to a previous value. Probably this value will be deleted (so doesn't cause an error).
Thanks,
Best,
J
Find below 2 queries.
The first query is the proposed solution.
This includes a function which you can't step through in the query editor.
The second query is an amended vesion for 1 product, with some lines commented out and some other lines added, so you can see the steps that are part of the function in the first query.
FYI: I first created the query for 1 product and turned this into a function for the final solution.
I tested with about 25,000 rows of data in Excel and it runs for about 2 seconds (!).
Proposed solution:
let
Source = TestData,
#"Grouped Rows" = Table.Group(Source, {"Product"}, {{"AllData", each fnPriceDifference(_), type table}}),
fnPriceDifference = (MyTable as table) as table =>
let
#"Sorted Rows" = Table.Buffer(Table.Sort(MyTable,{{"date", Order.Descending}})),
#"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1),
#"Added Index1" = Table.AddIndexColumn(#"Added Index", "Index.1", 1, 1),
#"Merged Queries" = Table.NestedJoin(#"Added Index1",{"Index.1"},#"Added Index1",{"Index"},"PreviousDate",JoinKind.LeftOuter),
#"Expanded PreviousDate" = Table.ExpandTableColumn(#"Merged Queries", "PreviousDate", {"Cost"}, {"PreviousDate.Cost"}),
#"Added Custom" = Table.AddColumn(#"Expanded PreviousDate", "Difference", each if [PreviousDate.Cost] = null then null else [Cost] - [PreviousDate.Cost]),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index", "Index.1", "PreviousDate.Cost"})
in
#"Removed Columns",
#"Expanded AllData" = Table.ExpandTableColumn(#"Grouped Rows", "AllData", {"date", "Campaign", "Cost", "id", "Difference"}, {"date", "Campaign", "Cost", "id", "Difference"})
in
#"Expanded AllData"
Code for 1 product only (just to see all steps):
let
Source = TestData,
// #"Grouped Rows" = Table.Group(Source, {"Product"}, {{"AllData", each fnPriceDifference(_), type table}}),
#"Grouped Rows" = Table.Group(Source, {"Product"}, {{"AllData", each _, type table}}),
MyTable = #"Grouped Rows"{[Product="A"]}[AllData],
// fnPriceDifference = (MyTable as table) as table =>
// let
#"Sorted Rows" = Table.Buffer(Table.Sort(MyTable,{{"date", Order.Descending}})),
#"Added Index" = Table.AddIndexColumn(#"Sorted Rows", "Index", 0, 1),
#"Added Index1" = Table.AddIndexColumn(#"Added Index", "Index.1", 1, 1),
#"Merged Queries" = Table.NestedJoin(#"Added Index1",{"Index.1"},#"Added Index1",{"Index"},"PreviousDate",JoinKind.LeftOuter),
#"Expanded PreviousDate" = Table.ExpandTableColumn(#"Merged Queries", "PreviousDate", {"Cost"}, {"PreviousDate.Cost"}),
#"Added Custom" = Table.AddColumn(#"Expanded PreviousDate", "Difference", each if [PreviousDate.Cost] = null then null else [Cost] - [PreviousDate.Cost]),
#"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"Index", "Index.1", "PreviousDate.Cost"})
// in
// #"Removed Columns",
// #"Expanded AllData" = Table.ExpandTableColumn(#"Grouped Rows", "AllData", {"date", "Campaign", "Cost", "id", "Difference"}, {"date", "Campaign", "Cost", "id", "Difference"})
in
// #"Expanded AllData"
#"Removed Columns"- Jmenas9 years agoAdvocate III
Hi MarcelBeug,
Your code helped me to resolve the issue. I just had to add a couple of extra steps to sort the rows correctly.
Thanks,
J