Forum Discussion

MarkDonald's avatar
MarkDonald
Regular Visitor
2 years ago
Solved

Repeat Friday Data for Weekends and Holidays

Hi - have searched and found similar problems, but no suggestions I have been able to follow...

 

I have data which is posted for each business day, while nothing at all appears for non business days.  I want to copy the data such that is is repeated for the following non business days, with the non business day date.

 

I have more than one row of data so a simple fill down isn't possible.  I can create another table with all dates, and merge with the original table to show the missing dates, but haven't managed to succesfully get any further than that.  I am pretty new to Power Query, so I might struggle with anything too complicated!

 

Any help is much appreciated.

 

My sample raw data is here: 

 

balanceDateConcatBalance
6/07/2023A:A:A0.00
6/07/2023A:A:B1.00
6/07/2023A:A:C2.00
6/07/2023A:A:D3.00
8/07/2023A:A:A0.00
8/07/2023A:A:B1.10
8/07/2023A:A:C2.20
8/07/2023A:A:D3.30
10/07/2023A:A:A0.00
10/07/2023A:A:B1.21
10/07/2023A:A:C2.42
10/07/2023A:A:D3.63

 

and the desired outcome is here:

 

balanceDateConcatBalance
6/07/2023A:A:A0.00
6/07/2023A:A:B1.00
6/07/2023A:A:C2.00
6/07/2023A:A:D3.00
7/07/2023A:A:A0.00
7/07/2023A:A:B1.00
7/07/2023A:A:C2.00
7/07/2023A:A:D3.00
8/07/2023A:A:A0.00
8/07/2023A:A:B1.10
8/07/2023A:A:C2.20
8/07/2023A:A:D3.30
9/07/2023A:A:A0.00
9/07/2023A:A:B1.10
9/07/2023A:A:C2.20
9/07/2023A:A:D3.30
10/07/2023A:A:A0.00
10/07/2023A:A:B1.21
10/07/2023A:A:C2.42
10/07/2023A:A:D3.63
  • ronrsnfld's avatar
    ronrsnfld
    2 years ago

    Here is another code that does not have the duplicates problem you ran into:

     

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("fc25DYAwEETRXja28B7IIDKOLiz33wYrjUjw2ppgghf8Wqlk3rKyGiU6D58/L8zUUo+Xv4zw9tcRPv724T5r/hFNiRFNjRFNAwrPop2iqhIrsqvGim4xau0F", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [balanceDate = _t, Concat = _t, Balance = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{
            {"balanceDate", type date}, {"Concat", type text}, {"Balance", Currency.Type}},"en-150"),
        
    //Group by date
        #"Grouped Rows" = Table.Group(#"Changed Type", {"balanceDate"}, {
            {"all", each _, type table [balanceDate=nullable date, Concat=nullable text, Balance=Currency.Type]}}),
    
    //Create table of All dates
        #"All Dates" = Table.FromColumns(
            {List.Dates(#"Grouped Rows"[balanceDate]{0},
                        Duration.Days(List.Last(#"Grouped Rows"[balanceDate])- #"Grouped Rows"[balanceDate]{0})+1,
                        #duration(1,0,0,0))},
                        type table[dates=date]),
    
    //Join the tables and sort so we have
    //  nulls where there are missing dates
        #"Join" = Table.Join(#"Grouped Rows","balanceDate",#"All Dates","dates",JoinKind.FullOuter),
        #"Sorted Rows" = Table.Sort(Join,{{"dates", Order.Ascending}}),
    
    //Replace nulls in balanceDate with the missing data
        #"Replace nulls" = Table.ReplaceValue(
            #"Sorted Rows",
            each [balanceDate],
            (r) as date=> if r[balanceDate]=null then r[dates] else r[balanceDate],
            Replacer.ReplaceValue,
            {"balanceDate"}
    
        ),
        #"Removed Columns" = Table.RemoveColumns(#"Replace nulls",{"dates"}),
        #"Filled Down" = Table.FillDown(#"Removed Columns",{"all"}),
        #"Expanded all" = Table.ExpandTableColumn(#"Filled Down", "all", {"Concat", "Balance"})
    in
        #"Expanded all"

     

     

     

     

     

     

10 Replies

  • Hello, MarkDonald 

    let
        Source = your_table,
        type_date = Table.TransformColumnTypes(Source,{{"balanceDate", type date}}),
        g = Table.Group(type_date, "balanceDate", {{"bd", each _}, {"data", each true}}),
        all_dates = List.Dates(
            List.Min(g[balanceDate]), 
            Duration.TotalDays(List.Max(g[balanceDate]) - List.Min(g[balanceDate])),
            #duration(1, 0, 0, 0)
        ),
        tl = Table.FromColumns({all_dates}, {"balanceDate"}),
        combine = g & tl,
        sort = Table.Sort(combine,{{"balanceDate", Order.Ascending}, {"data", Order.Descending}}),
        f_down = Table.FillDown(sort,{"bd"}),
        expand = Table.ExpandTableColumn(f_down, "bd", {"Concat", "Balance"})[[balanceDate], [Concat], [Balance]]
    in
        expand
    • MarkDonald's avatar
      MarkDonald
      Regular Visitor

      Thankyou so much!  Works perfectly, and I almost understand what it is doing 😁

       

  • MarkDonald in the end we just select the columns we need 

    [[balanceDate], [Concat], [Balance]]

     other than this it's all in the code. Just walk through steps in PQ editor. Refer to MS site to get understanding. 

  • MarkDonald's avatar
    MarkDonald
    Regular Visitor

    Actually, sorry, it isn't looking quite right, it seems to be duplicating for the dates that already exist, apart from the last.  I have stepped through and can see where it is duplicating the dates but can't really figure out how to make it work without this happening.  I can correct at the end by simply removing duplicates so I have something that works, so thanks!  This is how I inserted your code: 

     

    let
    Source = Excel.CurrentWorkbook(){[Name="Table17"]}[Content],
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"balanceDate", type datetime}, {"Concat", type text}, {"Balance", type number}}),

    type_date = Table.TransformColumnTypes(Source,{{"balanceDate", type date}}),
    g = Table.Group(type_date, "balanceDate", {{"bd", each _}, {"data", each true}}),
    all_dates = List.Dates(List.Min(g[balanceDate]), Duration.TotalDays(List.Max(g[balanceDate]) - List.Min(g[balanceDate])),#duration(1, 0, 0, 0)),
    tl = Table.FromColumns({all_dates}, {"balanceDate"}),
    combine = g & tl,
    sort = Table.Sort(combine,{{"balanceDate", Order.Ascending}, {"data", Order.Descending}}),
    f_down = Table.FillDown(sort,{"bd"}),
    expand = Table.ExpandTableColumn(f_down, "bd", {"Concat", "Balance"})[[balanceDate], [Concat], [Balance]]
    in
    expand

     

     

    balanceDateConcatBalance

    6/07/2023A:A:A0
    6/07/2023A:A:B1
    6/07/2023A:A:C2
    6/07/2023A:A:D3
    6/07/2023A:A:A0
    6/07/2023A:A:B1
    6/07/2023A:A:C2
    6/07/2023A:A:D3
    7/07/2023A:A:A0
    7/07/2023A:A:B1
    7/07/2023A:A:C2
    7/07/2023A:A:D3
    8/07/2023A:A:A0
    8/07/2023A:A:B1.1
    8/07/2023A:A:C2.2
    8/07/2023A:A:D3.3
    8/07/2023A:A:A0
    8/07/2023A:A:B1.1
    8/07/2023A:A:C2.2
    8/07/2023A:A:D3.3
    9/07/2023A:A:A0
    9/07/2023A:A:B1.1
    9/07/2023A:A:C2.2
    9/07/2023A:A:D3.3
    10/07/2023A:A:A0
    10/07/2023A:A:B1.21
    10/07/2023A:A:C2.42
    10/07/2023A:A:D3.63
    • ronrsnfld's avatar
      ronrsnfld
      Super User

      Here is another code that does not have the duplicates problem you ran into:

       

       

      let
          Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("fc25DYAwEETRXja28B7IIDKOLiz33wYrjUjw2ppgghf8Wqlk3rKyGiU6D58/L8zUUo+Xv4zw9tcRPv724T5r/hFNiRFNjRFNAwrPop2iqhIrsqvGim4xau0F", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [balanceDate = _t, Concat = _t, Balance = _t]),
          #"Changed Type" = Table.TransformColumnTypes(Source,{
              {"balanceDate", type date}, {"Concat", type text}, {"Balance", Currency.Type}},"en-150"),
          
      //Group by date
          #"Grouped Rows" = Table.Group(#"Changed Type", {"balanceDate"}, {
              {"all", each _, type table [balanceDate=nullable date, Concat=nullable text, Balance=Currency.Type]}}),
      
      //Create table of All dates
          #"All Dates" = Table.FromColumns(
              {List.Dates(#"Grouped Rows"[balanceDate]{0},
                          Duration.Days(List.Last(#"Grouped Rows"[balanceDate])- #"Grouped Rows"[balanceDate]{0})+1,
                          #duration(1,0,0,0))},
                          type table[dates=date]),
      
      //Join the tables and sort so we have
      //  nulls where there are missing dates
          #"Join" = Table.Join(#"Grouped Rows","balanceDate",#"All Dates","dates",JoinKind.FullOuter),
          #"Sorted Rows" = Table.Sort(Join,{{"dates", Order.Ascending}}),
      
      //Replace nulls in balanceDate with the missing data
          #"Replace nulls" = Table.ReplaceValue(
              #"Sorted Rows",
              each [balanceDate],
              (r) as date=> if r[balanceDate]=null then r[dates] else r[balanceDate],
              Replacer.ReplaceValue,
              {"balanceDate"}
      
          ),
          #"Removed Columns" = Table.RemoveColumns(#"Replace nulls",{"dates"}),
          #"Filled Down" = Table.FillDown(#"Removed Columns",{"all"}),
          #"Expanded all" = Table.ExpandTableColumn(#"Filled Down", "all", {"Concat", "Balance"})
      in
          #"Expanded all"

       

       

       

       

       

       

      • MarkDonald's avatar
        MarkDonald
        Regular Visitor
        Thanks - I am just having difficulty merging this into my code as I have already had to get it into the format posted above. At present I have (after removing a few steps): let Source = Sql.Databases("xx"), xxx = Source{[Name="xxx"]}[Data], findur_viewBalanceSheetHistoric = edw{[Schema="findur",Item="viewBalanceSheetHistoric"]}[Data], #"Filtered Rows" = Table.SelectRows(findur_viewBalanceSheetHistoric, each [balanceDate] >= #date(2023, 6, 29) and [balanceDate] <= #date(2023, 7, 17)), #"Filtered Rows1" = Table.SelectRows(#"Filtered Rows", each not Text.StartsWith([portfolio], "ZZZ")), #"Merged Columns" = Table.CombineColumns(#"Filtered Rows1",{"accountGroup", "reportPortfolio", "postingCurrency"},Combiner.CombineTextByDelimiter(":", QuoteStyle.None),"Concat"), #"Grouped Rows" = Table.Group(#"Merged Columns", {"balanceDate", "Concat"}, {{"Balance", each List.Sum([balanceBase]), type nullable number}}) in #"Grouped Rows"