Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
7 years ago

Need Help! Data transformation

Hi,

Could anyone please help me to transform data in desired output in Power BI.

 

Input Data

IDCOL1COL1_ValueCOL2COL2_ValueCOL3COL3_Value
1ServerNameABCDEFServerNamePOIUYTServerNameQWERTY
2DiskNameDDiskNameEDiskNameF
3ApplicationLKJApplicationHYUnullnull

 

Output:

IDServerNameDiskNameApplication
1ABCDEFDLKJ
2POIUYTEHYU
3QWERTYFnull

 

Thanks,

Randhir

19 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Anonymous,

    Please try the transformation below:

    let
        Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUQpOLSpLLfJLzE0FchydnF1c3dBFA/w9QyND0EUDw12DQiKVYnWilYyAXJfM4myojAsq1xWV6wbWYgyyraAgJzM5sSQzPw/I8/H2whDziAwFknmlOTkwKjYWAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ID = _t, COL1 = _t, COL1_Value = _t, COL2 = _t, COL2_Value = _t, COL3 = _t, COL3_Value = _t]),
        #"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}),
        #"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"ID"}),
        #"Transposed Table" = Table.Transpose(#"Removed Columns"),
        #"Promoted Headers" = Table.PromoteHeaders(#"Transposed Table", [PromoteAllScalars=true]),
        #"Changed Type1" = Table.TransformColumnTypes(#"Promoted Headers",{{"ServerName", type text}, {"DiskName", type text}, {"Application", type text}}),
        #"Removed Alternate Rows" = Table.AlternateRows(#"Changed Type1",1,1,1),
        #"Added Index" = Table.AddIndexColumn(#"Removed Alternate Rows", "ID", 1, 1),
        #"Reordered Columns" = Table.ReorderColumns(#"Added Index",{"ID", "ServerName", "DiskName", "Application"})
    in
        #"Reordered Columns"

     

  • smpa01's avatar
    smpa01
    Community Champion
    let
        Source = Web.Page(Web.Contents("https://community.powerbi.com/t5/Desktop/Need-Help-Data-transformation/m-p/649703#M311554")),
        Data0 = Source{0}[Data],
        #"Promoted Headers" = Table.PromoteHeaders(Data0, [PromoteAllScalars=true]),
        #"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}),
        #"Removed Other Columns" = Table.SelectColumns(#"Changed Type",{"ID", "COL1", "COL1_Value"}),
        Custom1 = Table.SelectColumns(#"Changed Type",{"ID", "COL2", "COL2_Value"}),
        #"Renamed Columns" = Table.RenameColumns(Custom1,{{"COL2", "COL1"}, {"COL2_Value", "COL1_Value"}}),
        Custom2 = Table.SelectColumns(#"Changed Type",{"ID", "COL3", "COL3_Value"}),
        #"Renamed Columns1" = Table.RenameColumns(Custom2,{{"COL3", "COL1"}, {"COL3_Value", "COL1_Value"}}),
        Custom3 = #"Removed Other Columns"&#"Renamed Columns"&#"Renamed Columns1",
        #"Counted Rows" = Table.RowCount(Custom3)/3,
        Custom4 = List.Repeat({"ServerName","DiskName","Application"}, #"Counted Rows"),
        Custom5 = Table.FromColumns(Table.ToColumns(Custom3)&{Custom4}),
        #"Removed Columns" = Table.RemoveColumns(Custom5,{"Column2"}),
        #"Added Index" = Table.AddIndexColumn(#"Removed Columns", "Index", 1, 1),
        #"Grouped Rows" = Table.Group(#"Added Index", {"Column4"}, {{"Count", each _, type table}}),
        #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each Table.AddIndexColumn([Count],"IX",1,1)),
        #"Removed Columns1" = Table.RemoveColumns(#"Added Custom",{"Count"}),
        #"Expanded Custom" = Table.ExpandTableColumn(#"Removed Columns1", "Custom", {"IX"}, {"IX"}),
        #"Removed Other Columns1" = Table.SelectColumns(#"Expanded Custom",{"IX"}),
        #"Removed Duplicates" = Table.Distinct(#"Removed Other Columns1"),
        #"Added Custom1" = Table.AddColumn(#"Removed Duplicates", "Custom", each {"ServerName", "DiskName","Application"}),
        #"Expanded Custom1" = Table.ExpandListColumn(#"Added Custom1", "Custom"),
        #"Added Index1" = Table.AddIndexColumn(#"Expanded Custom1", "Index", 1, 1),
        #"Merged Queries" = Table.NestedJoin(#"Added Index1",{"Custom", "Index"},#"Added Index",{"Column4", "Index"},"Added Index1",JoinKind.LeftOuter),
        #"Expanded Added Index1" = Table.ExpandTableColumn(#"Merged Queries", "Added Index1", {"Column3"}, {"Column3"}),
        #"Removed Columns2" = Table.RemoveColumns(#"Expanded Added Index1",{"Index"}),
        #"Pivoted Column" = Table.Pivot(#"Removed Columns2", List.Distinct(#"Removed Columns2"[Custom]), "Custom", "Column3"),
        #"Renamed Columns2" = Table.RenameColumns(#"Pivoted Column",{{"IX", "ID"}})
    in
        #"Renamed Columns2"
  • Anonymous's avatar
    Anonymous
    Not applicable

    The answers above seem tough to me...but I'm not an M guy. Just doing it using the user interface in Power Query:

     

    1. Import data as-is

    2. Remove columns ID, COL2 and COL3

    3. Highlight COL1 and use TRANSPOSE feature

    4. Use First Rows as Headers

     

    Does that help?

    Scott

     

    • parry2k's avatar
      parry2k
      Super User

      randhir I hope this will help, i'm sure previous solution will also work but I didn't tested those.

       

      let
          Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUQpOLSpLLfJLzE0FchydnF1c3dBFA/w9QyND0EUDw12DQiKVYnWilYyAXJfM4myojAsq1xWV6wbWYgyyraAgJzM5sSQzPw/I8/H2whDziAwFknmlOTkwKjYWAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ID = _t, COL1 = _t, COL1_Value = _t, COL2 = _t, COL2_Value = _t, COL3 = _t, COL3_Value = _t]),
          #"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}),
          #"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"COL2", "COL3"}),
          #"Unpivoted Only Selected Columns" = Table.Unpivot(#"Removed Columns", {"COL1_Value", "COL2_Value", "COL3_Value"}, "Attribute", "Value"),
          #"Removed Columns1" = Table.RemoveColumns(#"Unpivoted Only Selected Columns",{"ID"}),
          #"Pivoted Column" = Table.Pivot(#"Removed Columns1", List.Distinct(#"Removed Columns1"[COL1]), "COL1", "Value"),
          #"Removed Columns2" = Table.RemoveColumns(#"Pivoted Column",{"Attribute"}),
          #"Added Index" = Table.AddIndexColumn(#"Removed Columns2", "Index", 1, 1)
      in
          #"Added Index"

       

       

  • Anonymous's avatar
    Anonymous
    Not applicable

    Thanks Anonymous  and smpa01 for your help and solution.

    Your solution works well in most of the cases. But I have few records where COL2 and COL3 values are not in order same as COL1.

    For Example:

     

    IDCOL1COL1_ValueCOL2COL2_ValueCOL3COL3_Value
    1ServerNameABCDEFDiskNameEServerNameQWERTY
    2DiskNameDServerNamePOIUYTDiskNameF
    3ApplicationLKJApplicationHYU  

     

    Could you please check if this scenerio could also be handled.

    Thanks,

    Randhir

    • smpa01's avatar
      smpa01
      Community Champion
      Can you give me a full sample data set which contains all the scenarios. Once you provide will look into it.
    • v-lili6-msft's avatar
      v-lili6-msft
      Community Support

      HI, Anonymous 

      You may try this way:

      Duplicate the basic table twice.

      Then remove COL1 and COL1_Value for first table, and  remove COL2 and COL2_Value for second table, and  remove COL3 and COL3_Value for yhird table

      Then merge table1 with table2 by ID and COL column, expend then merge with table3 again by the same logic.

      Now we get the COL2 and COL3 values are in order same as COL1

       

      Then transform it as above.

       

      Best Regards,
      Lin

       

      • Anonymous's avatar
        Anonymous
        Not applicable

        Hi 

            SelectColumnFromTable1 = Table.SelectColumns(#"Changed Type",{"COL1","COL1_Value"}),
            SelectColumnFromTable2 = Table.SelectColumns(#"Changed Type_",{"COL2","COL2_Value"}),
        SelectColumnFromTable3 = Table.SelectColumns(#"Changed Type_",{"COL3","COL3_Value"}), #"Appended Query" = Table.Combine({SelectColumnFromTable1 , SelectColumnFromTable2, SelectColumnFromTable3 })

         

    • smpa01's avatar
      smpa01
      Community Champion

      Completely dynamic M

       

      let
          Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCk4tKkst8kvMTVWK1YlWcskszoZzHAsKcjKTE0sy8/OUYmMB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Keywords to be searched for" = _t]),
          #"Changed Type" = Table.TransformColumnTypes(Source,{{"Keywords to be searched for", type text}}),
          #"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Keywords to be searched for", "Keywords to be searched for"}})
      in
          #"Renamed Columns"

      Table4

       

      let
          Source = Web.Page(Web.Contents("https://community.powerbi.com/t5/Desktop/Need-Help-Data-transformation/m-p/650169#M311784")),
          Data2 = Source{2}[Data],
          #"Promoted Headers" = Table.PromoteHeaders(Data2, [PromoteAllScalars=true]),
          #"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}),
          Custom1 = Table.DemoteHeaders(#"Changed Type"),
          #"Changed Type1" = Table.TransformColumnTypes(Custom1,{{"Column1", type any}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}}),
          #"Removed Columns" = Table.RemoveColumns(#"Changed Type1",{"Column1"}),
          #"Transposed Table" = Table.Transpose(#"Removed Columns"),
          Custom2 = Table.AlternateRows(#"Transposed Table",1,1,1),
          #"Unpivoted Other Columns1" = Table.UnpivotOtherColumns(Custom2, {"Column1"}, "Attribute", "Value"),
          Custom3 = Table.AlternateRows(#"Transposed Table",0,1,1),
          #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Custom3, {"Column1"}, "Attribute", "Value"),
          #"Replaced Value" = Table.ReplaceValue(#"Unpivoted Other Columns","_Value","",Replacer.ReplaceText,{"Column1"}),
          #"Merged Queries" = Table.NestedJoin(#"Replaced Value",{"Column1", "Attribute"},#"Unpivoted Other Columns1",{"Column1", "Attribute"},"Replaced Value",JoinKind.LeftOuter),
          #"Expanded Replaced Value" = Table.ExpandTableColumn(#"Merged Queries", "Replaced Value", {"Attribute", "Value"}, {"Attribute.1", "Value.1"}),
          #"Removed Other Columns" = Table.SelectColumns(#"Expanded Replaced Value",{"Column1", "Value", "Value.1"}),
          #"Grouped Rows" = Table.Group(#"Removed Other Columns", {"Column1"}, {{"AD", each _, type table}}),
          #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each List.Accumulate(
      [AD][Value.1],
      "",
      (state,current)=> (state &" "& current)
      )),
          #"Removed Columns1" = Table.RemoveColumns(#"Added Custom",{"Column1"}),
          #"Expanded AD" = Table.ExpandTableColumn(#"Removed Columns1", "AD", {"Column1", "Value", "Value.1"}, {"Column1", "Value", "Value.1"}),
          #"Added Custom1" = Table.AddColumn(#"Expanded AD", "Custom.1", each let 
         CurrentText = [Custom],
         Result= Table.SelectRows(Table4,each not Text.Contains(CurrentText, [Keywords to be searched for], Comparer.OrdinalIgnoreCase))
      
      in
        Result),
          #"Expanded Custom.1" = Table.ExpandTableColumn(#"Added Custom1", "Custom.1", {"Keywords to be searched for"}, {"Keywords to be searched for"}),
          #"Added Custom2" = Table.AddColumn(#"Expanded Custom.1", "Custom.1", each if [Value.1]="" and [Keywords to be searched for]<>"" then [Keywords to be searched for] else [Value.1]),
          #"Removed Other Columns1" = Table.SelectColumns(#"Added Custom2",{"Column1", "Custom.1", "Value"}),
          #"Pivoted Column" = Table.Pivot(#"Removed Other Columns1", List.Distinct(#"Removed Other Columns1"[Custom.1]), "Custom.1", "Value")
      in
          #"Pivoted Column"

      Desired Output

      • Anonymous's avatar
        Anonymous
        Not applicable
        Hi smpa01, Thanks for your time.
        I am trying to understand your code. Could you please explain what is Table4 in query.
        Result= Table.SelectRows(Table4,each not Text.Contains(CurrentText, [Keywords to be searched for]...

        Actually "Added Custom1" step is failing with error "The name Table4 was not recognized".
        Please help me to understand and resolved.

        Thanks,
        Randhir Singh