Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
Anonymous
Not applicable

Need Help! Data transformation

Hi,

Could anyone please help me to transform data in desired output in Power BI.

 

Input Data

IDCOL1COL1_ValueCOL2COL2_ValueCOL3COL3_Value
1ServerNameABCDEFServerNamePOIUYTServerNameQWERTY
2DiskNameDDiskNameEDiskNameF
3ApplicationLKJApplicationHYUnullnull

 

Output:

IDServerNameDiskNameApplication
1ABCDEFDLKJ
2POIUYTEHYU
3QWERTYFnull

 

Thanks,

Randhir

19 REPLIES 19
Anonymous
Not applicable

Thanks @Anonymous  and @smpa01 for your help and solution.

Your solution works well in most of the cases. But I have few records where COL2 and COL3 values are not in order same as COL1.

For Example:

 

IDCOL1COL1_ValueCOL2COL2_ValueCOL3COL3_Value
1ServerNameABCDEFDiskNameEServerNameQWERTY
2DiskNameDServerNamePOIUYTDiskNameF
3ApplicationLKJApplicationHYU  

 

Could you please check if this scenerio could also be handled.

Thanks,

Randhir

Completely dynamic M

 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCk4tKkst8kvMTVWK1YlWcskszoZzHAsKcjKTE0sy8/OUYmMB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Keywords to be searched for" = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Keywords to be searched for", type text}}),
    #"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Keywords to be searched for", "Keywords to be searched for"}})
in
    #"Renamed Columns"

Table4

 

let
    Source = Web.Page(Web.Contents("https://community.powerbi.com/t5/Desktop/Need-Help-Data-transformation/m-p/650169#M311784")),
    Data2 = Source{2}[Data],
    #"Promoted Headers" = Table.PromoteHeaders(Data2, [PromoteAllScalars=true]),
    #"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}),
    Custom1 = Table.DemoteHeaders(#"Changed Type"),
    #"Changed Type1" = Table.TransformColumnTypes(Custom1,{{"Column1", type any}, {"Column2", type text}, {"Column3", type text}, {"Column4", type text}, {"Column5", type text}, {"Column6", type text}, {"Column7", type text}}),
    #"Removed Columns" = Table.RemoveColumns(#"Changed Type1",{"Column1"}),
    #"Transposed Table" = Table.Transpose(#"Removed Columns"),
    Custom2 = Table.AlternateRows(#"Transposed Table",1,1,1),
    #"Unpivoted Other Columns1" = Table.UnpivotOtherColumns(Custom2, {"Column1"}, "Attribute", "Value"),
    Custom3 = Table.AlternateRows(#"Transposed Table",0,1,1),
    #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Custom3, {"Column1"}, "Attribute", "Value"),
    #"Replaced Value" = Table.ReplaceValue(#"Unpivoted Other Columns","_Value","",Replacer.ReplaceText,{"Column1"}),
    #"Merged Queries" = Table.NestedJoin(#"Replaced Value",{"Column1", "Attribute"},#"Unpivoted Other Columns1",{"Column1", "Attribute"},"Replaced Value",JoinKind.LeftOuter),
    #"Expanded Replaced Value" = Table.ExpandTableColumn(#"Merged Queries", "Replaced Value", {"Attribute", "Value"}, {"Attribute.1", "Value.1"}),
    #"Removed Other Columns" = Table.SelectColumns(#"Expanded Replaced Value",{"Column1", "Value", "Value.1"}),
    #"Grouped Rows" = Table.Group(#"Removed Other Columns", {"Column1"}, {{"AD", each _, type table}}),
    #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each List.Accumulate(
[AD][Value.1],
"",
(state,current)=> (state &" "& current)
)),
    #"Removed Columns1" = Table.RemoveColumns(#"Added Custom",{"Column1"}),
    #"Expanded AD" = Table.ExpandTableColumn(#"Removed Columns1", "AD", {"Column1", "Value", "Value.1"}, {"Column1", "Value", "Value.1"}),
    #"Added Custom1" = Table.AddColumn(#"Expanded AD", "Custom.1", each let 
   CurrentText = [Custom],
   Result= Table.SelectRows(Table4,each not Text.Contains(CurrentText, [Keywords to be searched for], Comparer.OrdinalIgnoreCase))

in
  Result),
    #"Expanded Custom.1" = Table.ExpandTableColumn(#"Added Custom1", "Custom.1", {"Keywords to be searched for"}, {"Keywords to be searched for"}),
    #"Added Custom2" = Table.AddColumn(#"Expanded Custom.1", "Custom.1", each if [Value.1]="" and [Keywords to be searched for]<>"" then [Keywords to be searched for] else [Value.1]),
    #"Removed Other Columns1" = Table.SelectColumns(#"Added Custom2",{"Column1", "Custom.1", "Value"}),
    #"Pivoted Column" = Table.Pivot(#"Removed Other Columns1", List.Distinct(#"Removed Other Columns1"[Custom.1]), "Custom.1", "Value")
in
    #"Pivoted Column"

Desired Output

Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs
Anonymous
Not applicable

Hi @smpa01, Thanks for your time.
I am trying to understand your code. Could you please explain what is Table4 in query.
Result= Table.SelectRows(Table4,each not Text.Contains(CurrentText, [Keywords to be searched for]...

Actually "Added Custom1" step is failing with error "The name Table4 was not recognized".
Please help me to understand and resolved.

Thanks,
Randhir Singh

You need to create a query called Table4 with the following code for the output to execute

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WCk4tKkst8kvMTVWK1YlWcskszoZzHAsKcjKTE0sy8/OUYmMB", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [#"Keywords to be searched for" = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Keywords to be searched for", type text}}),
    #"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Keywords to be searched for", "Keywords to be searched for"}})
in
    #"Renamed Columns"
Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs
Anonymous
Not applicable

Hi @smpa01,

Thanks for your time and efforts. This is what I am looking for a dynamic approach. Actually I have total total 40 columns ( COL1,COL_Value, COL2,COL2_Value.... COL20, COL20_Value) .

Your solution is working fine as per sample data posted on web.

But still I am not able to understand how TABLE4 works.

It would be really great if you can explain me working of TABLE4.

Sorry,I am new to Power BI, could you please consider the table below as your Excel source and provide me query.

I'll modify your query for others remaining columns and try at my real data. Please note, given below sample data is containing all the possible scenerios.

IDCOL1COL1_ValueCOL2COL2_ValueCOL3COL3_ValueCOL4COL4_ValueCOL5COL5_Value
1ServerTypestr6valApplicationstr12valMetricstr18valPerimeterstr22valSubServerTypestr1val
2ServerTypestr7valServerTypestr13valMetricstr19valPerimeterstr23valSubServerTypestr2val
3ServerNamestr8valServerNamestr14valApplicationstr20valMetricstr24valSubServerTypestr3val
4ServerNamestr9valServerNamestr15valServerTypestr21valApplicationstr25valPerimeterstr4val
5ServerNamestr10valServerNamestr16val    Metricstr5val
6DiskNamestr11valServerNamestr17val    Metricstr6val

 

Thanks,

Randhir

 

Hi,

 

This M code generates the table below

 

let
    Source = Excel.CurrentWorkbook(){[Name="Data"]}[Content],
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}, {"COL4", type text}, {"COL4_Value", type text}, {"COL5", type text}, {"COL5_Value", type text}}),
    #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Changed Type", {"ID"}, "Attribute", "Value"),
    #"Uppercased Text" = Table.TransformColumns(#"Unpivoted Other Columns",{{"Attribute", Text.Upper, type text}}),
    #"Added Custom" = Table.AddColumn(#"Uppercased Text", "Custom", each if Text.End([Attribute],5)="VALUE" then [Value] else null),
    #"Filled Down" = Table.FillDown(#"Added Custom",{"Custom"}),
    #"Added Custom1" = Table.AddColumn(#"Filled Down", "Custom.1", each if Text.End([Attribute],5)="VALUE" then null else [Value]),
    #"Filled Down1" = Table.FillDown(#"Added Custom1",{"Custom.1"}),
    #"Filtered Rows" = Table.SelectRows(#"Filled Down1", each ([Custom] <> null)),
    #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"Attribute", "Value"}),
    #"Added Custom2" = Table.AddColumn(#"Removed Columns", "Custom.2", each Number.ToText([ID])&[Custom.1]),
    Partition = Table.Group(#"Added Custom2", {"Custom.2"}, {{"Partition", each Table.AddIndexColumn(_, "Index",1,1), type table}}),
    #"Expanded Partition" = Table.ExpandTableColumn(Partition, "Partition", {"ID", "Custom", "Custom.1", "Index"}, {"ID", "Custom", "Custom.1", "Index"}),
    #"Removed Columns1" = Table.RemoveColumns(#"Expanded Partition",{"Custom.2"}),
    #"Added Custom3" = Table.AddColumn(#"Removed Columns1", "Custom.2", each [Custom.1]&Number.ToText([Index])),
    #"Removed Columns2" = Table.RemoveColumns(#"Added Custom3",{"Custom.1", "Index"}),
    #"Sorted Rows" = Table.Sort(#"Removed Columns2",{{"ID", Order.Ascending}, {"Custom.2", Order.Ascending}}),
    #"Pivoted Column" = Table.Pivot(#"Sorted Rows", List.Distinct(#"Sorted Rows"[Custom.2]), "Custom.2", "Custom")
in
    #"Pivoted Column"

Untitled.png


Regards,
Ashish Mathur
http://www.ashishmathur.com
https://www.linkedin.com/in/excelenthusiasts/

Table4 is created to replicate this -

https://community.powerbi.com/t5/Desktop/Power-query-Add-column-with-list-of-keywords-found-in-text/...

 

Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs

That's how it would look once you create two queries with the codes that I provided you.Table 2 is the desired output.

sn1.JPG

 

Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs

@Anonymous just wondering if you tried the code I posted. 



Subscribe to the @PowerBIHowTo YT channel for an upcoming video on List and Record functions in Power Query!!

Learn Power BI and Fabric - subscribe to our YT channel - Click here: @PowerBIHowTo

If my solution proved useful, I'd be delighted to receive Kudos. When you put effort into asking a question, it's equally thoughtful to acknowledge and give Kudos to the individual who helped you solve the problem. It's a small gesture that shows appreciation and encouragement! ❤


Did I answer your question? Mark my post as a solution. Proud to be a Super User! Appreciate your Kudos 🙂
Feel free to email me with any of your BI needs.

tex628
Community Champion
Community Champion

Do you want the ID column to just work as an index column in the final table? 



Connect on LinkedIn
Anonymous
Not applicable

Hi @tex628, ID is just for index.

HI, @Anonymous 

You may try this way:

Duplicate the basic table twice.

Then remove COL1 and COL1_Value for first table, and  remove COL2 and COL2_Value for second table, and  remove COL3 and COL3_Value for yhird table

3.JPG4.JPG5.JPG

Then merge table1 with table2 by ID and COL column, expend then merge with table3 again by the same logic.

Now we get the COL2 and COL3 values are in order same as COL1

6.JPG

 

Then transform it as above.

7.JPG

 

Best Regards,
Lin

 

Community Support Team _ Lin
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Anonymous
Not applicable

Hi 

    SelectColumnFromTable1 = Table.SelectColumns(#"Changed Type",{"COL1","COL1_Value"}),
    SelectColumnFromTable2 = Table.SelectColumns(#"Changed Type_",{"COL2","COL2_Value"}),
SelectColumnFromTable3 = Table.SelectColumns(#"Changed Type_",{"COL3","COL3_Value"}), #"Appended Query" = Table.Combine({SelectColumnFromTable1 , SelectColumnFromTable2, SelectColumnFromTable3 })

 

hi, @Anonymous 

You could close "Enable load" for other queries.

For example:

6.JPG

Then they won't load in data model.

7.JPG

 

Best Regards,

Lin

Community Support Team _ Lin
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Can you give me a full sample data set which contains all the scenarios. Once you provide will look into it.
Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs
Anonymous
Not applicable

The answers above seem tough to me...but I'm not an M guy. Just doing it using the user interface in Power Query:

 

1. Import data as-is

2. Remove columns ID, COL2 and COL3

3. Highlight COL1 and use TRANSPOSE feature

4. Use First Rows as Headers

 

Does that help?

Scott

 

@randhir I hope this will help, i'm sure previous solution will also work but I didn't tested those.

 

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUQpOLSpLLfJLzE0FchydnF1c3dBFA/w9QyND0EUDw12DQiKVYnWilYyAXJfM4myojAsq1xWV6wbWYgyyraAgJzM5sSQzPw/I8/H2whDziAwFknmlOTkwKjYWAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ID = _t, COL1 = _t, COL1_Value = _t, COL2 = _t, COL2_Value = _t, COL3 = _t, COL3_Value = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}),
    #"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"COL2", "COL3"}),
    #"Unpivoted Only Selected Columns" = Table.Unpivot(#"Removed Columns", {"COL1_Value", "COL2_Value", "COL3_Value"}, "Attribute", "Value"),
    #"Removed Columns1" = Table.RemoveColumns(#"Unpivoted Only Selected Columns",{"ID"}),
    #"Pivoted Column" = Table.Pivot(#"Removed Columns1", List.Distinct(#"Removed Columns1"[COL1]), "COL1", "Value"),
    #"Removed Columns2" = Table.RemoveColumns(#"Pivoted Column",{"Attribute"}),
    #"Added Index" = Table.AddIndexColumn(#"Removed Columns2", "Index", 1, 1)
in
    #"Added Index"

 

 



Subscribe to the @PowerBIHowTo YT channel for an upcoming video on List and Record functions in Power Query!!

Learn Power BI and Fabric - subscribe to our YT channel - Click here: @PowerBIHowTo

If my solution proved useful, I'd be delighted to receive Kudos. When you put effort into asking a question, it's equally thoughtful to acknowledge and give Kudos to the individual who helped you solve the problem. It's a small gesture that shows appreciation and encouragement! ❤


Did I answer your question? Mark my post as a solution. Proud to be a Super User! Appreciate your Kudos 🙂
Feel free to email me with any of your BI needs.

smpa01
Super User
Super User

let
    Source = Web.Page(Web.Contents("https://community.powerbi.com/t5/Desktop/Need-Help-Data-transformation/m-p/649703#M311554")),
    Data0 = Source{0}[Data],
    #"Promoted Headers" = Table.PromoteHeaders(Data0, [PromoteAllScalars=true]),
    #"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}),
    #"Removed Other Columns" = Table.SelectColumns(#"Changed Type",{"ID", "COL1", "COL1_Value"}),
    Custom1 = Table.SelectColumns(#"Changed Type",{"ID", "COL2", "COL2_Value"}),
    #"Renamed Columns" = Table.RenameColumns(Custom1,{{"COL2", "COL1"}, {"COL2_Value", "COL1_Value"}}),
    Custom2 = Table.SelectColumns(#"Changed Type",{"ID", "COL3", "COL3_Value"}),
    #"Renamed Columns1" = Table.RenameColumns(Custom2,{{"COL3", "COL1"}, {"COL3_Value", "COL1_Value"}}),
    Custom3 = #"Removed Other Columns"&#"Renamed Columns"&#"Renamed Columns1",
    #"Counted Rows" = Table.RowCount(Custom3)/3,
    Custom4 = List.Repeat({"ServerName","DiskName","Application"}, #"Counted Rows"),
    Custom5 = Table.FromColumns(Table.ToColumns(Custom3)&{Custom4}),
    #"Removed Columns" = Table.RemoveColumns(Custom5,{"Column2"}),
    #"Added Index" = Table.AddIndexColumn(#"Removed Columns", "Index", 1, 1),
    #"Grouped Rows" = Table.Group(#"Added Index", {"Column4"}, {{"Count", each _, type table}}),
    #"Added Custom" = Table.AddColumn(#"Grouped Rows", "Custom", each Table.AddIndexColumn([Count],"IX",1,1)),
    #"Removed Columns1" = Table.RemoveColumns(#"Added Custom",{"Count"}),
    #"Expanded Custom" = Table.ExpandTableColumn(#"Removed Columns1", "Custom", {"IX"}, {"IX"}),
    #"Removed Other Columns1" = Table.SelectColumns(#"Expanded Custom",{"IX"}),
    #"Removed Duplicates" = Table.Distinct(#"Removed Other Columns1"),
    #"Added Custom1" = Table.AddColumn(#"Removed Duplicates", "Custom", each {"ServerName", "DiskName","Application"}),
    #"Expanded Custom1" = Table.ExpandListColumn(#"Added Custom1", "Custom"),
    #"Added Index1" = Table.AddIndexColumn(#"Expanded Custom1", "Index", 1, 1),
    #"Merged Queries" = Table.NestedJoin(#"Added Index1",{"Custom", "Index"},#"Added Index",{"Column4", "Index"},"Added Index1",JoinKind.LeftOuter),
    #"Expanded Added Index1" = Table.ExpandTableColumn(#"Merged Queries", "Added Index1", {"Column3"}, {"Column3"}),
    #"Removed Columns2" = Table.RemoveColumns(#"Expanded Added Index1",{"Index"}),
    #"Pivoted Column" = Table.Pivot(#"Removed Columns2", List.Distinct(#"Removed Columns2"[Custom]), "Custom", "Column3"),
    #"Renamed Columns2" = Table.RenameColumns(#"Pivoted Column",{{"IX", "ID"}})
in
    #"Renamed Columns2"
Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs
Anonymous
Not applicable

@Anonymous,

Please try the transformation below:

let
    Source = Table.FromRows(Json.Document(Binary.Decompress(Binary.FromText("i45WMlTSUQpOLSpLLfJLzE0FchydnF1c3dBFA/w9QyND0EUDw12DQiKVYnWilYyAXJfM4myojAsq1xWV6wbWYgyyraAgJzM5sSQzPw/I8/H2whDziAwFknmlOTkwKjYWAA==", BinaryEncoding.Base64), Compression.Deflate)), let _t = ((type text) meta [Serialized.Text = true]) in type table [ID = _t, COL1 = _t, COL1_Value = _t, COL2 = _t, COL2_Value = _t, COL3 = _t, COL3_Value = _t]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"ID", Int64.Type}, {"COL1", type text}, {"COL1_Value", type text}, {"COL2", type text}, {"COL2_Value", type text}, {"COL3", type text}, {"COL3_Value", type text}}),
    #"Removed Columns" = Table.RemoveColumns(#"Changed Type",{"ID"}),
    #"Transposed Table" = Table.Transpose(#"Removed Columns"),
    #"Promoted Headers" = Table.PromoteHeaders(#"Transposed Table", [PromoteAllScalars=true]),
    #"Changed Type1" = Table.TransformColumnTypes(#"Promoted Headers",{{"ServerName", type text}, {"DiskName", type text}, {"Application", type text}}),
    #"Removed Alternate Rows" = Table.AlternateRows(#"Changed Type1",1,1,1),
    #"Added Index" = Table.AddIndexColumn(#"Removed Alternate Rows", "ID", 1, 1),
    #"Reordered Columns" = Table.ReorderColumns(#"Added Index",{"ID", "ServerName", "DiskName", "Application"})
in
    #"Reordered Columns"

 

Helpful resources

Announcements
LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.