Forum Discussion
Repeat Friday Data for Weekends and Holidays
Hi - have searched and found similar problems, but no suggestions I have been able to follow...
I have data which is posted for each business day, while nothing at all appears for non business days. I want to copy the data such that is is repeated for the following non business days, with the non business day date.
I have more than one row of data so a simple fill down isn't possible. I can create another table with all dates, and merge with the original table to show the missing dates, but haven't managed to succesfully get any further than that. I am pretty new to Power Query, so I might struggle with anything too complicated!
Any help is much appreciated.
My sample raw data is here:
| balanceDate | Concat | Balance |
| 6/07/2023 | A:A:A | 0.00 |
| 6/07/2023 | A:A:B | 1.00 |
| 6/07/2023 | A:A:C | 2.00 |
| 6/07/2023 | A:A:D | 3.00 |
| 8/07/2023 | A:A:A | 0.00 |
| 8/07/2023 | A:A:B | 1.10 |
| 8/07/2023 | A:A:C | 2.20 |
| 8/07/2023 | A:A:D | 3.30 |
| 10/07/2023 | A:A:A | 0.00 |
| 10/07/2023 | A:A:B | 1.21 |
| 10/07/2023 | A:A:C | 2.42 |
| 10/07/2023 | A:A:D | 3.63 |
and the desired outcome is here:
| balanceDate | Concat | Balance |
| 6/07/2023 | A:A:A | 0.00 |
| 6/07/2023 | A:A:B | 1.00 |
| 6/07/2023 | A:A:C | 2.00 |
| 6/07/2023 | A:A:D | 3.00 |
| 7/07/2023 | A:A:A | 0.00 |
| 7/07/2023 | A:A:B | 1.00 |
| 7/07/2023 | A:A:C | 2.00 |
| 7/07/2023 | A:A:D | 3.00 |
| 8/07/2023 | A:A:A | 0.00 |
| 8/07/2023 | A:A:B | 1.10 |
| 8/07/2023 | A:A:C | 2.20 |
| 8/07/2023 | A:A:D | 3.30 |
| 9/07/2023 | A:A:A | 0.00 |
| 9/07/2023 | A:A:B | 1.10 |
| 9/07/2023 | A:A:C | 2.20 |
| 9/07/2023 | A:A:D | 3.30 |
| 10/07/2023 | A:A:A | 0.00 |
| 10/07/2023 | A:A:B | 1.21 |
| 10/07/2023 | A:A:C | 2.42 |
| 10/07/2023 | A:A:D | 3.63 |
Here is another code that does not have the duplicates problem you ran into:
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("fc25DYAwEETRXja28B7IIDKOLiz33wYrjUjw2ppgghf8Wqlk3rKyGiU6D58/L8zUUo+Xv4zw9tcRPv724T5r/hFNiRFNjRFNAwrPop2iqhIrsqvGim4xau0F", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [balanceDate = _t, Concat = _t, Balance = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{ {"balanceDate", type date}, {"Concat", type text}, {"Balance", Currency.Type}},"en-150"), //Group by date #"Grouped Rows" = Table.Group(#"Changed Type", {"balanceDate"}, { {"all", each _, type table [balanceDate=nullable date, Concat=nullable text, Balance=Currency.Type]}}), //Create table of All dates #"All Dates" = Table.FromColumns( {List.Dates(#"Grouped Rows"[balanceDate]{0}, Duration.Days(List.Last(#"Grouped Rows"[balanceDate])- #"Grouped Rows"[balanceDate]{0})+1, #duration(1,0,0,0))}, type table[dates=date]), //Join the tables and sort so we have // nulls where there are missing dates #"Join" = Table.Join(#"Grouped Rows","balanceDate",#"All Dates","dates",JoinKind.FullOuter), #"Sorted Rows" = Table.Sort(Join,{{"dates", Order.Ascending}}), //Replace nulls in balanceDate with the missing data #"Replace nulls" = Table.ReplaceValue( #"Sorted Rows", each [balanceDate], (r) as date=> if r[balanceDate]=null then r[dates] else r[balanceDate], Replacer.ReplaceValue, {"balanceDate"} ), #"Removed Columns" = Table.RemoveColumns(#"Replace nulls",{"dates"}), #"Filled Down" = Table.FillDown(#"Removed Columns",{"all"}), #"Expanded all" = Table.ExpandTableColumn(#"Filled Down", "all", {"Concat", "Balance"}) in #"Expanded all"
10 Replies
- AlienSxSuper User
Hello, MarkDonald
let Source = your_table, type_date = Table.TransformColumnTypes(Source,{{"balanceDate", type date}}), g = Table.Group(type_date, "balanceDate", {{"bd", each _}, {"data", each true}}), all_dates = List.Dates( List.Min(g[balanceDate]), Duration.TotalDays(List.Max(g[balanceDate]) - List.Min(g[balanceDate])), #duration(1, 0, 0, 0) ), tl = Table.FromColumns({all_dates}, {"balanceDate"}), combine = g & tl, sort = Table.Sort(combine,{{"balanceDate", Order.Ascending}, {"data", Order.Descending}}), f_down = Table.FillDown(sort,{"bd"}), expand = Table.ExpandTableColumn(f_down, "bd", {"Concat", "Balance"})[[balanceDate], [Concat], [Balance]] in expand- MarkDonaldRegular Visitor
Thankyou so much! Works perfectly, and I almost understand what it is doing 😁
- AlienSxSuper User
MarkDonald in the end we just select the columns we need
[[balanceDate], [Concat], [Balance]]other than this it's all in the code. Just walk through steps in PQ editor. Refer to MS site to get understanding.
- MarkDonaldRegular Visitor
Actually, sorry, it isn't looking quite right, it seems to be duplicating for the dates that already exist, apart from the last. I have stepped through and can see where it is duplicating the dates but can't really figure out how to make it work without this happening. I can correct at the end by simply removing duplicates so I have something that works, so thanks! This is how I inserted your code:
let
Source = Excel.CurrentWorkbook(){[Name="Table17"]}[Content],
#"Changed Type" = Table.TransformColumnTypes(Source,{{"balanceDate", type datetime}, {"Concat", type text}, {"Balance", type number}}),type_date = Table.TransformColumnTypes(Source,{{"balanceDate", type date}}),
g = Table.Group(type_date, "balanceDate", {{"bd", each _}, {"data", each true}}),
all_dates = List.Dates(List.Min(g[balanceDate]), Duration.TotalDays(List.Max(g[balanceDate]) - List.Min(g[balanceDate])),#duration(1, 0, 0, 0)),
tl = Table.FromColumns({all_dates}, {"balanceDate"}),
combine = g & tl,
sort = Table.Sort(combine,{{"balanceDate", Order.Ascending}, {"data", Order.Descending}}),
f_down = Table.FillDown(sort,{"bd"}),
expand = Table.ExpandTableColumn(f_down, "bd", {"Concat", "Balance"})[[balanceDate], [Concat], [Balance]]
in
expandbalanceDateConcatBalance
6/07/2023 A:A:A 0 6/07/2023 A:A:B 1 6/07/2023 A:A:C 2 6/07/2023 A:A:D 3 6/07/2023 A:A:A 0 6/07/2023 A:A:B 1 6/07/2023 A:A:C 2 6/07/2023 A:A:D 3 7/07/2023 A:A:A 0 7/07/2023 A:A:B 1 7/07/2023 A:A:C 2 7/07/2023 A:A:D 3 8/07/2023 A:A:A 0 8/07/2023 A:A:B 1.1 8/07/2023 A:A:C 2.2 8/07/2023 A:A:D 3.3 8/07/2023 A:A:A 0 8/07/2023 A:A:B 1.1 8/07/2023 A:A:C 2.2 8/07/2023 A:A:D 3.3 9/07/2023 A:A:A 0 9/07/2023 A:A:B 1.1 9/07/2023 A:A:C 2.2 9/07/2023 A:A:D 3.3 10/07/2023 A:A:A 0 10/07/2023 A:A:B 1.21 10/07/2023 A:A:C 2.42 10/07/2023 A:A:D 3.63 - ronrsnfldSuper User
Here is another code that does not have the duplicates problem you ran into:
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("fc25DYAwEETRXja28B7IIDKOLiz33wYrjUjw2ppgghf8Wqlk3rKyGiU6D58/L8zUUo+Xv4zw9tcRPv724T5r/hFNiRFNjRFNAwrPop2iqhIrsqvGim4xau0F", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [balanceDate = _t, Concat = _t, Balance = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{ {"balanceDate", type date}, {"Concat", type text}, {"Balance", Currency.Type}},"en-150"), //Group by date #"Grouped Rows" = Table.Group(#"Changed Type", {"balanceDate"}, { {"all", each _, type table [balanceDate=nullable date, Concat=nullable text, Balance=Currency.Type]}}), //Create table of All dates #"All Dates" = Table.FromColumns( {List.Dates(#"Grouped Rows"[balanceDate]{0}, Duration.Days(List.Last(#"Grouped Rows"[balanceDate])- #"Grouped Rows"[balanceDate]{0})+1, #duration(1,0,0,0))}, type table[dates=date]), //Join the tables and sort so we have // nulls where there are missing dates #"Join" = Table.Join(#"Grouped Rows","balanceDate",#"All Dates","dates",JoinKind.FullOuter), #"Sorted Rows" = Table.Sort(Join,{{"dates", Order.Ascending}}), //Replace nulls in balanceDate with the missing data #"Replace nulls" = Table.ReplaceValue( #"Sorted Rows", each [balanceDate], (r) as date=> if r[balanceDate]=null then r[dates] else r[balanceDate], Replacer.ReplaceValue, {"balanceDate"} ), #"Removed Columns" = Table.RemoveColumns(#"Replace nulls",{"dates"}), #"Filled Down" = Table.FillDown(#"Removed Columns",{"all"}), #"Expanded all" = Table.ExpandTableColumn(#"Filled Down", "all", {"Concat", "Balance"}) in #"Expanded all"- MarkDonaldRegular VisitorThanks - I am just having difficulty merging this into my code as I have already had to get it into the format posted above. At present I have (after removing a few steps): let Source = Sql.Databases("xx"), xxx = Source{[Name="xxx"]}[Data], findur_viewBalanceSheetHistoric = edw{[Schema="findur",Item="viewBalanceSheetHistoric"]}[Data], #"Filtered Rows" = Table.SelectRows(findur_viewBalanceSheetHistoric, each [balanceDate] >= #date(2023, 6, 29) and [balanceDate] <= #date(2023, 7, 17)), #"Filtered Rows1" = Table.SelectRows(#"Filtered Rows", each not Text.StartsWith([portfolio], "ZZZ")), #"Merged Columns" = Table.CombineColumns(#"Filtered Rows1",{"accountGroup", "reportPortfolio", "postingCurrency"},Combiner.CombineTextByDelimiter(":", QuoteStyle.None),"Concat"), #"Grouped Rows" = Table.Group(#"Merged Columns", {"balanceDate", "Concat"}, {{"Balance", each List.Sum([balanceBase]), type nullable number}}) in #"Grouped Rows"