Forum Discussion
Survey data from Microsoft form
- 8 months ago
You should compare your approach with this:
Unpivot the data
Remove the trailing "spaces" and digits from the resultant Attributes column.
Pivot the Attributes column, using a custom function to avoid the usual errors seen when you have multiple entries per aggregation:
Custom Pivot Function
Rename as seen in the code
//credit: Cam Wallace https://www.dingbatdata.com/2018/03/08/non-aggregate-pivot-with-multiple-rows-in-powerquery/ //Rename: fnPivotAll (Source as table, ColToPivot as text, ColForValues as text)=> let PivotColNames = List.Buffer(List.Distinct(Table.Column(Source,ColToPivot))), #"Pivoted Column" = Table.Pivot(Source, PivotColNames, ColToPivot, ColForValues, each _), TableFromRecordOfLists = (rec as record, fieldnames as list) => let PartialRecord = Record.SelectFields(rec,fieldnames), RecordToList = Record.ToList(PartialRecord), Table = Table.FromColumns(RecordToList,fieldnames) in Table, #"Added Custom" = Table.AddColumn(#"Pivoted Column", "Values", each TableFromRecordOfLists(_,PivotColNames)), #"Removed Other Columns" = Table.RemoveColumns(#"Added Custom",PivotColNames), #"Expanded Values" = Table.ExpandTableColumn(#"Removed Other Columns", "Values", PivotColNames) in #"Expanded Values"Main Code
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content], #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Source, {"ID","Task"}, "Attribute", "Value"), #"Remove Trailing Digits and Spaces" = Table.TransformColumns(#"Unpivoted Other Columns", {"Attribute", each Text.TrimEnd(_,{"0".."9"," "}), type text}), #"Pivot No Aggregation" = fnPivotAll(#"Remove Trailing Digits and Spaces","Attribute","Value"), #"Changed Type" = Table.TransformColumnTypes(#"Pivot No Aggregation",{ {"ID", Int64.Type}, {"Task", type text}, {"city", type text}, {"Type of Business", type text}, {"Education need", type text}, {"Total", Int64.Type}, {"When", type text}, {"Degree", type text}}) in #"Changed Type"Results from your data:
- 8 months ago
Hi HaiVN , Thank you for reaching out to the Microsoft Community Forum.
I normalized the form output in Power Query so each education need becomes a separate row instead of duplicating the dataset. This makes the model scalable and suitable for analysis by education type, degree, and timing. The transformation uses unpivot/pivot logic and requires no manual maintenance if more education needs are added.
To help you better understand the implementation, I’ve attached the .pbix file for your reference. Please take a look at it and let me know your observations.
- 8 months ago
HI HaiVN just change the source with your actual table.
Here is complete m-code
let Source = Table.FromRows( { {1, "Project A", "Hanoi", "Manufacturing", "Data Science", 5, "Q1 2025", "Master", "AI/ML", 3, "Q2 2025", "PhD", "Cybersecurity", 2, "Q3 2025", "Bachelor"} }, type table [ ID = Int64.Type, Task = text, city = text, #"Type of Business" = text, #"Education need 1" = text, #"Total 1" = Int64.Type, #"When 1" = text, #"Degree 1" = text, #"Education need 2" = text, #"Total 2" = Int64.Type, #"When 2" = text, #"Degree 2" = text, #"Education need 3" = text, #"Total 3" = Int64.Type, #"When 3" = text, #"Degree 3" = text ] ), FixedColumns = {"ID", "Task", "city", "Type of Business"}, Unpivoted = Table.UnpivotOtherColumns(Source, FixedColumns, "Attribute", "Value"), AddBaseName = Table.AddColumn(Unpivoted, "BaseName", each Text.TrimEnd(Text.BeforeDelimiter([Attribute], " ", {0, RelativePosition.FromEnd}), {" "})), AddGroupNum = Table.AddColumn(AddBaseName, "GroupNum", each Text.AfterDelimiter([Attribute], " ", {0, RelativePosition.FromEnd})), Filtered = Table.SelectRows(AddGroupNum, each [Value] <> null and [Value] <> ""), Cleaned = Table.RemoveColumns(Filtered, {"Attribute"}), Pivoted = Table.Pivot(Cleaned, List.Distinct(Cleaned[BaseName]), "BaseName", "Value"), Final = Table.RemoveColumns(Pivoted, {"GroupNum"}) in FinalThanks
Hi HaiVN , Hope you are doing well. Kindly let us know if the issue has been resolved or if further assistance is needed. Your input could be helpful to others in the community.