Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.
Hi,
Could anyone please help me to transform data in desired output in Power BI.
Input Data
ID | COL1 | COL1_Value | COL2 | COL2_Value | COL3 | COL3_Value |
1 | ServerName | ABCDEF | ServerName | POIUYT | ServerName | QWERTY |
2 | DiskName | D | DiskName | E | DiskName | F |
3 | Application | LKJ | Application | HYU | null | null |
Output:
ID | ServerName | DiskName | Application |
1 | ABCDEF | D | LKJ |
2 | POIUYT | E | HYU |
3 | QWERTY | F | null |
Thanks,
Randhir
Thanks @Anonymous and @smpa01 for your help and solution.
Your solution works well in most of the cases. But I have few records where COL2 and COL3 values are not in order same as COL1.
For Example:
ID | COL1 | COL1_Value | COL2 | COL2_Value | COL3 | COL3_Value |
1 | ServerName | ABCDEF | DiskName | E | ServerName | QWERTY |
2 | DiskName | D | ServerName | POIUYT | DiskName | F |
3 | Application | LKJ | Application | HYU |
Could you please check if this scenerio could also be handled.
Thanks,
Randhir
Completely dynamic M
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCk4tKkst8kvMTVWK1YlWcskszoZzHAsKcjKTE0sy8/OUYmMB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Keywords to be searched for" = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Keywords to be searched for", type text}}), #"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Keywords to be searched for", "Keywords to be searched for"}}) in #"Renamed Columns"
Table4
let Source = Web.Page(Web.Contents("https://community.powerbi.com/t5/Desktop/Need-Help-Data-transformation/m-p/650169#M311784")), Data2 = Source{2}[Data], #"Promoted Headers" = Table.PromoteHeaders(Data2, [PromoteAllScalars=true]), #"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}), Custom1 = Table.DemoteHeaders(#"Changed Type"), #"Changed Type1" = Table.TransformColumnTypes(Custom1,{{"Column1", type any}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}}), #"Removed Columns" = Table.RemoveColumns(#"Changed Type1",{"Column1"}), #"Transposed Table" = Table.Transpose(#"Removed Columns"), Custom2 = Table.AlternateRows(#"Transposed Table",1,1,1), #"Unpivoted Other Columns1" = Table.UnpivotOtherColumns(Custom2, {"Column1"}, "Attribute", "Value"), Custom3 = Table.AlternateRows(#"Transposed Table",0,1,1), #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Custom3, {"Column1"}, "Attribute", "Value"), #"Replaced Value" = Table.ReplaceValue(#"Unpivoted Other Columns","_Value","",Replacer.ReplaceText,{"Column1"}), #"Merged Queries" = Table.NestedJoin(#"Replaced Value",{"Column1", "Attribute"},#"Unpivoted Other Columns1",{"Column1", "Attribute"},"Replaced Value",JoinKind.LeftOuter), #"Expanded Replaced Value" = Table.ExpandTableColumn(#"Merged Queries", "Replaced Value", {"Attribute", "Value"}, {"Attribute.1", "Value.1"}), #"Removed Other Columns" = Table.SelectColumns(#"Expanded Replaced Value",{"Column1", "Value", "Value.1"}), #"Grouped Rows" = Table.Group(#"Removed Other Columns", {"Column1"}, {{"AD", each _, type table}}), #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each List.Accumulate( [AD][Value.1], "", (state,current)=> (state &" "& current) )), #"Removed Columns1" = Table.RemoveColumns(#"Added Custom",{"Column1"}), #"Expanded AD" = Table.ExpandTableColumn(#"Removed Columns1", "AD", {"Column1", "Value", "Value.1"}, {"Column1", "Value", "Value.1"}), #"Added Custom1" = Table.AddColumn(#"Expanded AD", "Custom.1", each let CurrentText = [Custom], Result= Table.SelectRows(Table4,each not Text.Contains(CurrentText, [Keywords to be searched for], Comparer.OrdinalIgnoreCase)) in Result), #"Expanded Custom.1" = Table.ExpandTableColumn(#"Added Custom1", "Custom.1", {"Keywords to be searched for"}, {"Keywords to be searched for"}), #"Added Custom2" = Table.AddColumn(#"Expanded Custom.1", "Custom.1", each if [Value.1]="" and [Keywords to be searched for]<>"" then [Keywords to be searched for] else [Value.1]), #"Removed Other Columns1" = Table.SelectColumns(#"Added Custom2",{"Column1", "Custom.1", "Value"}), #"Pivoted Column" = Table.Pivot(#"Removed Other Columns1", List.Distinct(#"Removed Other Columns1"[Custom.1]), "Custom.1", "Value") in #"Pivoted Column"
Desired Output
You need to create a query called Table4 with the following code for the output to execute
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCk4tKkst8kvMTVWK1YlWcskszoZzHAsKcjKTE0sy8/OUYmMB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Keywords to be searched for" = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"Keywords to be searched for", type text}}), #"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Keywords to be searched for", "Keywords to be searched for"}}) in #"Renamed Columns"
Hi @smpa01,
Thanks for your time and efforts. This is what I am looking for a dynamic approach. Actually I have total total 40 columns ( COL1,COL_Value, COL2,COL2_Value.... COL20, COL20_Value) .
Your solution is working fine as per sample data posted on web.
But still I am not able to understand how TABLE4 works.
It would be really great if you can explain me working of TABLE4.
Sorry,I am new to Power BI, could you please consider the table below as your Excel source and provide me query.
I'll modify your query for others remaining columns and try at my real data. Please note, given below sample data is containing all the possible scenerios.
ID | COL1 | COL1_Value | COL2 | COL2_Value | COL3 | COL3_Value | COL4 | COL4_Value | COL5 | COL5_Value |
1 | ServerType | str6val | Application | str12val | Metric | str18val | Perimeter | str22val | SubServerType | str1val |
2 | ServerType | str7val | ServerType | str13val | Metric | str19val | Perimeter | str23val | SubServerType | str2val |
3 | ServerName | str8val | ServerName | str14val | Application | str20val | Metric | str24val | SubServerType | str3val |
4 | ServerName | str9val | ServerName | str15val | ServerType | str21val | Application | str25val | Perimeter | str4val |
5 | ServerName | str10val | ServerName | str16val | Metric | str5val | ||||
6 | DiskName | str11val | ServerName | str17val | Metric | str6val |
Thanks,
Randhir
Hi,
This M code generates the table below
let Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content], #"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}, {"COL4", type text}, {"COL4_Value", type text}, {"COL5", type text}, {"COL5_Value", type text}}), #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"ID"}, "Attribute", "Value"), #"Uppercased Text" = Table.TransformColumns(#"Unpivoted Other Columns",{{"Attribute", Text.Upper, type text}}), #"Added Custom" = Table.AddColumn(#"Uppercased Text", "Custom", each if Text.End([Attribute],5)="VALUE" then [Value] else null), #"Filled Down" = Table.FillDown(#"Added Custom",{"Custom"}), #"Added Custom1" = Table.AddColumn(#"Filled Down", "Custom.1", each if Text.End([Attribute],5)="VALUE" then null else [Value]), #"Filled Down1" = Table.FillDown(#"Added Custom1",{"Custom.1"}), #"Filtered Rows" = Table.SelectRows(#"Filled Down1", each ([Custom] <> null)), #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Attribute", "Value"}), #"Added Custom2" = Table.AddColumn(#"Removed Columns", "Custom.2", each Number.ToText([ID])&[Custom.1]), Partition = Table.Group(#"Added Custom2", {"Custom.2"}, {{"Partition", each Table.AddIndexColumn(_, "Index",1,1), type table}}), #"Expanded Partition" = Table.ExpandTableColumn(Partition, "Partition", {"ID", "Custom", "Custom.1", "Index"}, {"ID", "Custom", "Custom.1", "Index"}), #"Removed Columns1" = Table.RemoveColumns(#"Expanded Partition",{"Custom.2"}), #"Added Custom3" = Table.AddColumn(#"Removed Columns1", "Custom.2", each [Custom.1]&Number.ToText([Index])), #"Removed Columns2" = Table.RemoveColumns(#"Added Custom3",{"Custom.1", "Index"}), #"Sorted Rows" = Table.Sort(#"Removed Columns2",{{"ID", Order.Ascending}, {"Custom.2", Order.Ascending}}), #"Pivoted Column" = Table.Pivot(#"Sorted Rows", List.Distinct(#"Sorted Rows"[Custom.2]), "Custom.2", "Custom") in #"Pivoted Column"
Table4 is created to replicate this -
That's how it would look once you create two queries with the codes that I provided you.Table 2 is the desired output.
@Anonymous just wondering if you tried the code I posted.
Subscribe to the @PowerBIHowTo YT channel for an upcoming video on List and Record functions in Power Query!!
Learn Power BI and Fabric - subscribe to our YT channel - Click here: @PowerBIHowTo
If my solution proved useful, I'd be delighted to receive Kudos. When you put effort into asking a question, it's equally thoughtful to acknowledge and give Kudos to the individual who helped you solve the problem. It's a small gesture that shows appreciation and encouragement! ❤
Did I answer your question? Mark my post as a solution. Proud to be a Super User! Appreciate your Kudos 🙂
Feel free to email me with any of your BI needs.
Do you want the ID column to just work as an index column in the final table?
HI, @Anonymous
You may try this way:
Duplicate the basic table twice.
Then remove COL1 and COL1_Value for first table, and remove COL2 and COL2_Value for second table, and remove COL3 and COL3_Value for yhird table
Then merge table1 with table2 by ID and COL column, expend then merge with table3 again by the same logic.
Now we get the COL2 and COL3 values are in order same as COL1
Then transform it as above.
Best Regards,
Lin
Hi @v-lili6-msft , Thanks for your help. I have total total 40 columns ( COL1,COL_Value, COL2,COL2_Value.... COL20, COL20_Value) .
I LIKE your approach. It is simple. Do you think it would be good to create 20 tables from performance and storage point of view?
Can we handle of this manual approach using M query? Like
SelectColumnFromTable1 = Table.SelectColumns(#"Changed Type",{"COL1","COL1_Value"}), SelectColumnFromTable2 = Table.SelectColumns(#"Changed Type_",{"COL2","COL2_Value"}),
SelectColumnFromTable3 = Table.SelectColumns(#"Changed Type_",{"COL3","COL3_Value"}), #"Appended Query" = Table.Combine({SelectColumnFromTable1 , SelectColumnFromTable2, SelectColumnFromTable3 })
And then PIVOT on "Appended Query"
Please advice on this.
Thanks,
Randhir Singh
hi, @Anonymous
You could close "Enable load" for other queries.
For example:
Then they won't load in data model.
Best Regards,
Lin
The answers above seem tough to me...but I'm not an M guy. Just doing it using the user interface in Power Query:
1. Import data as-is
2. Remove columns ID, COL2 and COL3
3. Highlight COL1 and use TRANSPOSE feature
4. Use First Rows as Headers
Does that help?
Scott
@randhir I hope this will help, i'm sure previous solution will also work but I didn't tested those.
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUQpOLSpLLfJLzE0FchydnF1c3dBFA/w9QyND0EUDw12DQiKVYnWilYyAXJfM4myojAsq1xWV6wbWYgyyraAgJzM5sSQzPw/I8/H2whDziAwFknmlOTkwKjYWAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ID = _t, COL1 = _t, COL1_Value = _t, COL2 = _t, COL2_Value = _t, COL3 = _t, COL3_Value = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}), #"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"COL2", "COL3"}), #"Unpivoted Only Selected Columns" = Table.Unpivot(#"Removed Columns", {"COL1_Value", "COL2_Value", "COL3_Value"}, "Attribute", "Value"), #"Removed Columns1" = Table.RemoveColumns(#"Unpivoted Only Selected Columns",{"ID"}), #"Pivoted Column" = Table.Pivot(#"Removed Columns1", List.Distinct(#"Removed Columns1"[COL1]), "COL1", "Value"), #"Removed Columns2" = Table.RemoveColumns(#"Pivoted Column",{"Attribute"}), #"Added Index" = Table.AddIndexColumn(#"Removed Columns2", "Index", 1, 1) in #"Added Index"
Subscribe to the @PowerBIHowTo YT channel for an upcoming video on List and Record functions in Power Query!!
Learn Power BI and Fabric - subscribe to our YT channel - Click here: @PowerBIHowTo
If my solution proved useful, I'd be delighted to receive Kudos. When you put effort into asking a question, it's equally thoughtful to acknowledge and give Kudos to the individual who helped you solve the problem. It's a small gesture that shows appreciation and encouragement! ❤
Did I answer your question? Mark my post as a solution. Proud to be a Super User! Appreciate your Kudos 🙂
Feel free to email me with any of your BI needs.
let Source = Web.Page(Web.Contents("https://community.powerbi.com/t5/Desktop/Need-Help-Data-transformation/m-p/649703#M311554")), Data0 = Source{0}[Data], #"Promoted Headers" = Table.PromoteHeaders(Data0, [PromoteAllScalars=true]), #"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}), #"Removed Other Columns" = Table.SelectColumns(#"Changed Type",{"ID", "COL1", "COL1_Value"}), Custom1 = Table.SelectColumns(#"Changed Type",{"ID", "COL2", "COL2_Value"}), #"Renamed Columns" = Table.RenameColumns(Custom1,{{"COL2", "COL1"}, {"COL2_Value", "COL1_Value"}}), Custom2 = Table.SelectColumns(#"Changed Type",{"ID", "COL3", "COL3_Value"}), #"Renamed Columns1" = Table.RenameColumns(Custom2,{{"COL3", "COL1"}, {"COL3_Value", "COL1_Value"}}), Custom3 = #"Removed Other Columns"&#"Renamed Columns"&#"Renamed Columns1", #"Counted Rows" = Table.RowCount(Custom3)/3, Custom4 = List.Repeat({"ServerName","DiskName","Application"}, #"Counted Rows"), Custom5 = Table.FromColumns(Table.ToColumns(Custom3)&{Custom4}), #"Removed Columns" = Table.RemoveColumns(Custom5,{"Column2"}), #"Added Index" = Table.AddIndexColumn(#"Removed Columns", "Index", 1, 1), #"Grouped Rows" = Table.Group(#"Added Index", {"Column4"}, {{"Count", each _, type table}}), #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each Table.AddIndexColumn([Count],"IX",1,1)), #"Removed Columns1" = Table.RemoveColumns(#"Added Custom",{"Count"}), #"Expanded Custom" = Table.ExpandTableColumn(#"Removed Columns1", "Custom", {"IX"}, {"IX"}), #"Removed Other Columns1" = Table.SelectColumns(#"Expanded Custom",{"IX"}), #"Removed Duplicates" = Table.Distinct(#"Removed Other Columns1"), #"Added Custom1" = Table.AddColumn(#"Removed Duplicates", "Custom", each {"ServerName", "DiskName","Application"}), #"Expanded Custom1" = Table.ExpandListColumn(#"Added Custom1", "Custom"), #"Added Index1" = Table.AddIndexColumn(#"Expanded Custom1", "Index", 1, 1), #"Merged Queries" = Table.NestedJoin(#"Added Index1",{"Custom", "Index"},#"Added Index",{"Column4", "Index"},"Added Index1",JoinKind.LeftOuter), #"Expanded Added Index1" = Table.ExpandTableColumn(#"Merged Queries", "Added Index1", {"Column3"}, {"Column3"}), #"Removed Columns2" = Table.RemoveColumns(#"Expanded Added Index1",{"Index"}), #"Pivoted Column" = Table.Pivot(#"Removed Columns2", List.Distinct(#"Removed Columns2"[Custom]), "Custom", "Column3"), #"Renamed Columns2" = Table.RenameColumns(#"Pivoted Column",{{"IX", "ID"}}) in #"Renamed Columns2"
@Anonymous,
Please try the transformation below:
let Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUQpOLSpLLfJLzE0FchydnF1c3dBFA/w9QyND0EUDw12DQiKVYnWilYyAXJfM4myojAsq1xWV6wbWYgyyraAgJzM5sSQzPw/I8/H2whDziAwFknmlOTkwKjYWAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ID = _t, COL1 = _t, COL1_Value = _t, COL2 = _t, COL2_Value = _t, COL3 = _t, COL3_Value = _t]), #"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}), #"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"ID"}), #"Transposed Table" = Table.Transpose(#"Removed Columns"), #"Promoted Headers" = Table.PromoteHeaders(#"Transposed Table", [PromoteAllScalars=true]), #"Changed Type1" = Table.TransformColumnTypes(#"Promoted Headers",{{"ServerName", type text}, {"DiskName", type text}, {"Application", type text}}), #"Removed Alternate Rows" = Table.AlternateRows(#"Changed Type1",1,1,1), #"Added Index" = Table.AddIndexColumn(#"Removed Alternate Rows", "ID", 1, 1), #"Reordered Columns" = Table.ReorderColumns(#"Added Index",{"ID", "ServerName", "DiskName", "Application"}) in #"Reordered Columns"