Forum Discussion

Jaf017's avatar
Jaf017
Advocate III
6 years ago
Solved

Data Translation

I am needing to translate my data from vertical to horizontal.

Is there a way to do this in Power Query?

 

Example pictures below.

 

  • Here is one way to do it.  To see how it works, just create a blank query, go to Advanced Editor, and replace the text there with the M code below.

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("hY47CoAwEESvElIr7m42iUmpWCripxLvfw0HLMQP2EzxhnnMtllWEmVfsi3sOCxIV4mvhIQMh+wVZJ27ie9NzBJBuLZ7cUlSAmvap0S/JYoiE5mxNxTQIF6yxyVMJLuEyZ8Rz857+wE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [GRP = _t, USER_INPTS = _t, OBC_TS = _t, ACK_BY = _t, ACK_TS = _t, ERROR = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"GRP", type text}, {"USER_INPTS", type text}, {"OBC_TS", type text}, {"ACK_BY", type text}, {"ACK_TS", type text}, {"ERROR", Int64.Type}}),
        #"Grouped Rows" = Table.Group(#"Changed Type", {"GRP"}, {{"AllRows", each _, type table [GRP=text, USER_INPTS=text, OBC_TS=text, ACK_BY=text, ACK_TS=text, ERROR=number]}}),
        #"Added Custom" = Table.AddColumn(#"Grouped Rows", "WithIndex", each Table.AddIndexColumn([AllRows], "Index",1,1)),
        #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"AllRows"}),
        #"Expanded WithIndex" = Table.ExpandTableColumn(#"Removed Columns", "WithIndex", {"USER_INPTS", "OBC_TS", "ACK_BY", "ACK_TS", "ERROR", "Index"}, {"USER_INPTS", "OBC_TS", "ACK_BY", "ACK_TS", "ERROR", "Index"}),
        #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Expanded WithIndex", {"GRP", "Index"}, "Attribute", "Value"),
        #"Merged Columns" = Table.CombineColumns(Table.TransformColumnTypes(#"Unpivoted Other Columns", {{"Index", type text}}, "en-US"),{"Attribute", "Index"},Combiner.CombineTextByDelimiter("_", QuoteStyle.None),"Merged"),
        #"Pivoted Column" = Table.Pivot(#"Merged Columns", List.Distinct(#"Merged Columns"[Merged]), "Merged", "Value"),
        #"Changed Type1" = Table.TransformColumnTypes(#"Pivoted Column",{{"GRP", type text}, {"USER_INPTS_1", type text}, {"OBC_TS_1", type datetime}, {"ACK_BY_1", type text}, {"ACK_TS_1", type text}, {"ERROR_1", Int64.Type}, {"USER_INPTS_2", type text}, {"OBC_TS_2", type text}, {"ACK_BY_2", type text}, {"ACK_TS_2", type text}, {"ERROR_2", Int64.Type}})
    in
        #"Changed Type1"

     

    If this works for you, please mark it as the solution.  Kudos are appreciated too.  Please let me know if not.

    Regards,

    Pat

5 Replies

  • mahoneypat's avatar
    mahoneypat
    Microsoft Employee

    Here is one way to do it.  To see how it works, just create a blank query, go to Advanced Editor, and replace the text there with the M code below.

     

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("hY47CoAwEESvElIr7m42iUmpWCripxLvfw0HLMQP2EzxhnnMtllWEmVfsi3sOCxIV4mvhIQMh+wVZJ27ie9NzBJBuLZ7cUlSAmvap0S/JYoiE5mxNxTQIF6yxyVMJLuEyZ8Rz857+wE=", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type nullable text) meta [Serialized.Text = true]) in type table [GRP = _t, USER_INPTS = _t, OBC_TS = _t, ACK_BY = _t, ACK_TS = _t, ERROR = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"GRP", type text}, {"USER_INPTS", type text}, {"OBC_TS", type text}, {"ACK_BY", type text}, {"ACK_TS", type text}, {"ERROR", Int64.Type}}),
        #"Grouped Rows" = Table.Group(#"Changed Type", {"GRP"}, {{"AllRows", each _, type table [GRP=text, USER_INPTS=text, OBC_TS=text, ACK_BY=text, ACK_TS=text, ERROR=number]}}),
        #"Added Custom" = Table.AddColumn(#"Grouped Rows", "WithIndex", each Table.AddIndexColumn([AllRows], "Index",1,1)),
        #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"AllRows"}),
        #"Expanded WithIndex" = Table.ExpandTableColumn(#"Removed Columns", "WithIndex", {"USER_INPTS", "OBC_TS", "ACK_BY", "ACK_TS", "ERROR", "Index"}, {"USER_INPTS", "OBC_TS", "ACK_BY", "ACK_TS", "ERROR", "Index"}),
        #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Expanded WithIndex", {"GRP", "Index"}, "Attribute", "Value"),
        #"Merged Columns" = Table.CombineColumns(Table.TransformColumnTypes(#"Unpivoted Other Columns", {{"Index", type text}}, "en-US"),{"Attribute", "Index"},Combiner.CombineTextByDelimiter("_", QuoteStyle.None),"Merged"),
        #"Pivoted Column" = Table.Pivot(#"Merged Columns", List.Distinct(#"Merged Columns"[Merged]), "Merged", "Value"),
        #"Changed Type1" = Table.TransformColumnTypes(#"Pivoted Column",{{"GRP", type text}, {"USER_INPTS_1", type text}, {"OBC_TS_1", type datetime}, {"ACK_BY_1", type text}, {"ACK_TS_1", type text}, {"ERROR_1", Int64.Type}, {"USER_INPTS_2", type text}, {"OBC_TS_2", type text}, {"ACK_BY_2", type text}, {"ACK_TS_2", type text}, {"ERROR_2", Int64.Type}})
    in
        #"Changed Type1"

     

    If this works for you, please mark it as the solution.  Kudos are appreciated too.  Please let me know if not.

    Regards,

    Pat

    • Jaf017's avatar
      Jaf017
      Advocate III

      This worked perfectly for the sample dataset!

      Thank you for this mahoneypat !

       

      I'm trying to implement this code into my dataset of 3.2M rows with up to 6 errors per GRP.

      Is this method scalable to that level?

      • mahoneypat's avatar
        mahoneypat
        Microsoft Employee

        It will probably be very slow with that many records.  Please report back if so. There may be some tricks to try.  In any case, what type of analysis are you doing with the data in that structure.  Typically, it is best to unpivot vs. pivot stuff out, so if possible I would encourage you to revist your data model structure.

         

        Regards,

        Pat