Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
5 years ago
Solved

Combining eight columns into four

Hi, I have not found the exact answer for this yet, so would be happy with some help. I have data is spread over eight columns, but I would like to have it in one long list. I cannot figure out how to do it in PowerQuery so I'd gladly appreciate some help. My data looks like this now:

 

ValueDateValue_1Date_1Value_2Date_2Value_3Date_3
100001-01-2020100401-01-2020100801-02-2020101201-03-2020
100101-01-2020100501-02-2020100901-02-2020101301-03-2020
100201-01-2020100601-02-2020101001-02-2020101401-03-2020
100301-01-2020100701-02-2020101101-02-2020101501-03-2020

 

And I would need it to look like this:

 

ValueDate
100101-01-2020
100201-01-2020
100301-01-2020
100401-01-2020
100501-02-2020
100601-02-2020
100701-02-2020
1008.......01-02-2020......

Thanks in advance 🙂 

  • Anonymous 

    Solution I:

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bcxbCoAwDETRveS7wiRtfayldP/bsGpB0Av5yYG5rZlLsmTyZVwormdYAdunxWse0/JjPd1Fh3X9r3VAMWMxoLjCWmAFixmKG6wdrH6K/QQ=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Value = _t, Date = _t, Value_1 = _t, Date_1 = _t, Value_2 = _t, Date_2 = _t, Value_3 = _t, Date_3 = _t]),
        #"Unpivoted Columns" = Table.UnpivotOtherColumns(Source, {}, "Attribute", "V"),
        #"Removed Columns" = Table.RemoveColumns(#"Unpivoted Columns",{"Attribute"}),
        #"Added Index" = Table.AddIndexColumn(#"Removed Columns", "Index", 0, 1, Int64.Type),
        #"Integer-Divided Column" = Table.TransformColumns(#"Added Index", {{"Index", each Number.IntegerDivide(_, 2), Int64.Type}}),
        #"Grouped Rows" = Table.Group(#"Integer-Divided Column", {"Index"}, {{"All", each Text.Combine(_[V],"#(tab)")}}),
        #"Removed Columns1" = Table.RemoveColumns(#"Grouped Rows",{"Index"}),
        #"Split Column by Delimiter" = Table.SplitColumn(#"Removed Columns1", "All", Splitter.SplitTextByDelimiter("#(tab)", QuoteStyle.Csv), {"Value", "Date"}),
        #"Sorted Rows" = Table.Sort(#"Split Column by Delimiter",{{"Value", Order.Ascending}})
    in
        #"Sorted Rows"

     

    Solution II, which borrowed Jimmy801 's elegant solution but with a bit amendment,

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bcxbCoAwDETRveS7wiRtfayldP/bsGpB0Av5yYG5rZlLsmTyZVwormdYAdunxWse0/JjPd1Fh3X9r3VAMWMxoLjCWmAFixmKG6wdrH6K/QQ=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Value = _t, Date = _t, Value_1 = _t, Date_1 = _t, Value_2 = _t, Date_2 = _t, Value_3 = _t, Date_3 = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"Value", Int64.Type}, {"Date", type date}, {"Value_1", Int64.Type}, {"Date_1", type date}, {"Value_2", Int64.Type}, {"Date_2", type date}, {"Value_3", Int64.Type}, {"Date_3", type date}}),
    
        ListSplit = List.Split(Table.ColumnNames(#"Changed Type"), 2),
        CreateTableGroups = List.Transform(ListSplit,
            each Table.FromColumns(List.Accumulate({0..List.Count(_)-1}, {}, (s,c)=>s&{Table.Column(#"Changed Type", _{c})}), {"Date","Value"})
        ),
        TableCombine = Table.Combine(CreateTableGroups)
    in
        TableCombine

     

  • CNENFRNL's avatar
    CNENFRNL
    5 years ago

    Hi, Anonymous , you might follow these steps,

    1. Import your own actual data into PQ;
    2. Copy any one of solutions proposed here ( code after the step "Source");
    3. Open up Advanced Editor, append the code proposed after your own step "Source";
  • Jimmy801's avatar
    Jimmy801
    5 years ago

    Hello Anonymous 

     

    you have to dig in the M-code to make things work. You can create your query (read from Workbook) and goto the advanved editor and replace your in statement with this code (Add a "," after your your code before pasting. Subsitute the #"Changed Type" with your last stepname. 

        ListSplit = List.Split(Table.ColumnNames(#"Changed Type"), 2),
        CreateTableGroups = List.Transform( ListSplit, (lst)=> Table.FromColumns({Table.Column(#"Changed Type", lst{0}),Table.Column(#"Changed Type", lst{1})}, {"Value", "Date"})),
        TableCombine = Table.Combine(CreateTableGroups)
    in
        TableCombine

     

    this is the most straightforward you can do. You can also use a more professional way to handle it, to create a new function like this Anonymous mentioned it already. Create a new blank query and copy paste this code there an name the query TransformYourTable

    (tbl as table) =>
    let
    
        ListSplit = List.Split(Table.ColumnNames(tbl), 2),
        CreateTableGroups = List.Transform( ListSplit, (lst)=> Table.FromColumns({Table.Column(tbl, lst{0}),Table.Column(tbl, lst{1})}, {"Value", "Date"})),
        TableCombine = Table.Combine(CreateTableGroups)
    in
        TableCombine

     

    then go to your query from the workbook, click on the table-symbol and select "invoke custom function"

    Right click on your last step and select "Insert Step After"

    go to the formula bar and put your new custom function

     

    Before

     

    After

     

    Hope this helps

     

    If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
    Kudoes are nice too

    Have fun

    Jimmy

     

10 Replies

  • Jimmy801's avatar
    Jimmy801
    Community Champion

    Hello Anonymous 

     

    check out this solution. It splits the table into pieces of 2 columns and change the column names to value and date. Afterwards the tables are combined again

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bcxbCoAwDETRveS7wiRtfayldP/bsGpB0Av5yYG5rZlLsmTyZVwormdYAdunxWse0/JjPd1Fh3X9r3VAMWMxoLjCWmAFixmKG6wdrH6K/QQ=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Value = _t, Date = _t, Value_1 = _t, Date_1 = _t, Value_2 = _t, Date_2 = _t, Value_3 = _t, Date_3 = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"Value", Int64.Type}, {"Date", type date}, {"Value_1", Int64.Type}, {"Date_1", type date}, {"Value_2", Int64.Type}, {"Date_2", type date}, {"Value_3", Int64.Type}, {"Date_3", type date}}),
        ListSplit = List.Split(Table.ColumnNames(#"Changed Type"), 2),
        CreateTableGroups = List.Transform( ListSplit, (lst)=> Table.FromColumns({Table.Column(#"Changed Type", lst{0}),Table.Column(#"Changed Type", lst{1})}, {"Value", "Date"})),
        TableCombine = Table.Combine(CreateTableGroups)
    in
        TableCombine

     

    Copy paste this code to the advanced editor in a new blank query to see how the solution works.

    If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
    Kudoes are nice too

    Have fun

    Jimmy

  • CNENFRNL's avatar
    CNENFRNL
    Community Champion

    Anonymous 

    Solution I:

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bcxbCoAwDETRveS7wiRtfayldP/bsGpB0Av5yYG5rZlLsmTyZVwormdYAdunxWse0/JjPd1Fh3X9r3VAMWMxoLjCWmAFixmKG6wdrH6K/QQ=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Value = _t, Date = _t, Value_1 = _t, Date_1 = _t, Value_2 = _t, Date_2 = _t, Value_3 = _t, Date_3 = _t]),
        #"Unpivoted Columns" = Table.UnpivotOtherColumns(Source, {}, "Attribute", "V"),
        #"Removed Columns" = Table.RemoveColumns(#"Unpivoted Columns",{"Attribute"}),
        #"Added Index" = Table.AddIndexColumn(#"Removed Columns", "Index", 0, 1, Int64.Type),
        #"Integer-Divided Column" = Table.TransformColumns(#"Added Index", {{"Index", each Number.IntegerDivide(_, 2), Int64.Type}}),
        #"Grouped Rows" = Table.Group(#"Integer-Divided Column", {"Index"}, {{"All", each Text.Combine(_[V],"#(tab)")}}),
        #"Removed Columns1" = Table.RemoveColumns(#"Grouped Rows",{"Index"}),
        #"Split Column by Delimiter" = Table.SplitColumn(#"Removed Columns1", "All", Splitter.SplitTextByDelimiter("#(tab)", QuoteStyle.Csv), {"Value", "Date"}),
        #"Sorted Rows" = Table.Sort(#"Split Column by Delimiter",{{"Value", Order.Ascending}})
    in
        #"Sorted Rows"

     

    Solution II, which borrowed Jimmy801 's elegant solution but with a bit amendment,

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bcxbCoAwDETRveS7wiRtfayldP/bsGpB0Av5yYG5rZlLsmTyZVwormdYAdunxWse0/JjPd1Fh3X9r3VAMWMxoLjCWmAFixmKG6wdrH6K/QQ=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Value = _t, Date = _t, Value_1 = _t, Date_1 = _t, Value_2 = _t, Date_2 = _t, Value_3 = _t, Date_3 = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"Value", Int64.Type}, {"Date", type date}, {"Value_1", Int64.Type}, {"Date_1", type date}, {"Value_2", Int64.Type}, {"Date_2", type date}, {"Value_3", Int64.Type}, {"Date_3", type date}}),
    
        ListSplit = List.Split(Table.ColumnNames(#"Changed Type"), 2),
        CreateTableGroups = List.Transform(ListSplit,
            each Table.FromColumns(List.Accumulate({0..List.Count(_)-1}, {}, (s,c)=>s&{Table.Column(#"Changed Type", _{c})}), {"Date","Value"})
        ),
        TableCombine = Table.Combine(CreateTableGroups)
    in
        TableCombine

     

    • Anonymous's avatar
      Anonymous
      Not applicable

      Hi CNENFRNL  , thank you, but maybe I was a bit unclear. What I meant is that, the data I showed in my example is just example data, and my real query where I have the data contains "real data" (and is a big file). So I would need to modify the code that Jimmy801  and mahoneypat kindly provided in order to 'plug' my source into the rest of the steps specified below. Since the source I have is an excel web file I'd need to paste my source somewhere in below code to make it work - but I cannot figure out where. 

      • CNENFRNL's avatar
        CNENFRNL
        Community Champion

        Hi, Anonymous , you might follow these steps,

        1. Import your own actual data into PQ;
        2. Copy any one of solutions proposed here ( code after the step "Source");
        3. Open up Advanced Editor, append the code proposed after your own step "Source";
  • mahoneypat's avatar
    mahoneypat
    Microsoft Employee

    Here is one way to do it with your example data.  To see how it works, just create a blank query, go to Advanced Editor, and replace the text there with the M code below.

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("bcxbCoAwDETRveS7wiRtfayldP/bsGpB0Av5yYG5rZlLsmTyZVwormdYAdunxWse0/JjPd1Fh3X9r3VAMWMxoLjCWmAFixmKG6wdrH6K/QQ=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [Value = _t, Date = _t, Value_1 = _t, Date_1 = _t, Value_2 = _t, Date_2 = _t, Value_3 = _t, Date_3 = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"Value", Int64.Type}, {"Date", type date}, {"Value_1", Int64.Type}, {"Date_1", type date}, {"Value_2", Int64.Type}, {"Date_2", type date}, {"Value_3", Int64.Type}, {"Date_3", type date}}),
        #"Added Index" = Table.AddIndexColumn(#"Changed Type", "Index", 1, 1, Int64.Type),
        #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Added Index", {"Index"}, "Attribute", "Value.1"),
        #"Extracted Text Before Delimiter" = Table.TransformColumns(#"Unpivoted Other Columns", {{"Attribute", each Text.BeforeDelimiter(_, "_"), type text}}),
        #"Added Index1" = Table.AddIndexColumn(#"Extracted Text Before Delimiter", "Index2", 1, 1, Int64.Type),
        #"Inserted Addition" = Table.AddColumn(#"Added Index1", "Addition", each Number.RoundUp([Index2] / 2,0), type number),
        #"Removed Columns2" = Table.RemoveColumns(#"Inserted Addition",{"Index", "Index2"}),
        #"Pivoted Column" = Table.Pivot(#"Removed Columns2", List.Distinct(#"Removed Columns2"[Attribute]), "Attribute", "Value.1"),
        #"Removed Columns" = Table.RemoveColumns(#"Pivoted Column",{"Addition"}),
        #"Changed Type1" = Table.TransformColumnTypes(#"Removed Columns",{{"Date", type date}, {"Value", Int64.Type}}),
        #"Sorted Rows" = Table.Sort(#"Changed Type1",{{"Value", Order.Ascending}})
    in
        #"Sorted Rows"

     

    Regards,

    Pat

  • wdx223_Daniel's avatar
    wdx223_Daniel
    Community Champion

    = Table.Combine(List.Transform(List.Split(Table.ToColumns(Source),2),each Table.FromColumns(_,{"Value","Date"})))

     

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi Jimmy801 mahoneypat thank you for your answers. Can you tell me which parts I would need to replace to make it work with my query (source)? My source is an excel file on Teams, so technically an excel online workbook. Thanks in advance. 

    • Jimmy801's avatar
      Jimmy801
      Community Champion

      Hello Anonymous 

       

      you have to dig in the M-code to make things work. You can create your query (read from Workbook) and goto the advanved editor and replace your in statement with this code (Add a "," after your your code before pasting. Subsitute the #"Changed Type" with your last stepname. 

          ListSplit = List.Split(Table.ColumnNames(#"Changed Type"), 2),
          CreateTableGroups = List.Transform( ListSplit, (lst)=> Table.FromColumns({Table.Column(#"Changed Type", lst{0}),Table.Column(#"Changed Type", lst{1})}, {"Value", "Date"})),
          TableCombine = Table.Combine(CreateTableGroups)
      in
          TableCombine

       

      this is the most straightforward you can do. You can also use a more professional way to handle it, to create a new function like this Anonymous mentioned it already. Create a new blank query and copy paste this code there an name the query TransformYourTable

      (tbl as table) =>
      let
      
          ListSplit = List.Split(Table.ColumnNames(tbl), 2),
          CreateTableGroups = List.Transform( ListSplit, (lst)=> Table.FromColumns({Table.Column(tbl, lst{0}),Table.Column(tbl, lst{1})}, {"Value", "Date"})),
          TableCombine = Table.Combine(CreateTableGroups)
      in
          TableCombine

       

      then go to your query from the workbook, click on the table-symbol and select "invoke custom function"

      Right click on your last step and select "Insert Step After"

      go to the formula bar and put your new custom function

       

      Before

       

      After

       

      Hope this helps

       

      If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
      Kudoes are nice too

      Have fun

      Jimmy

       

  • Anonymous's avatar
    Anonymous
    Not applicable

    Thank you all! I tried the version from Jimmy801 and it worked fine 🙂