Forum Discussion
Survey data from Microsoft form
- 8 months ago
You should compare your approach with this:
Unpivot the data
Remove the trailing "spaces" and digits from the resultant Attributes column.
Pivot the Attributes column, using a custom function to avoid the usual errors seen when you have multiple entries per aggregation:
Custom Pivot Function
Rename as seen in the code
//credit: Cam Wallace https://www.dingbatdata.com/2018/03/08/non-aggregate-pivot-with-multiple-rows-in-powerquery/ //Rename: fnPivotAll (Source as table, ColToPivot as text, ColForValues as text)=> let PivotColNames = List.Buffer(List.Distinct(Table.Column(Source,ColToPivot))), #"Pivoted Column" = Table.Pivot(Source, PivotColNames, ColToPivot, ColForValues, each _), TableFromRecordOfLists = (rec as record, fieldnames as list) => let PartialRecord = Record.SelectFields(rec,fieldnames), RecordToList = Record.ToList(PartialRecord), Table = Table.FromColumns(RecordToList,fieldnames) in Table, #"Added Custom" = Table.AddColumn(#"Pivoted Column", "Values", each TableFromRecordOfLists(_,PivotColNames)), #"Removed Other Columns" = Table.RemoveColumns(#"Added Custom",PivotColNames), #"Expanded Values" = Table.ExpandTableColumn(#"Removed Other Columns", "Values", PivotColNames) in #"Expanded Values"Main Code
let Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content], #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(Source, {"ID","Task"}, "Attribute", "Value"), #"Remove Trailing Digits and Spaces" = Table.TransformColumns(#"Unpivoted Other Columns", {"Attribute", each Text.TrimEnd(_,{"0".."9"," "}), type text}), #"Pivot No Aggregation" = fnPivotAll(#"Remove Trailing Digits and Spaces","Attribute","Value"), #"Changed Type" = Table.TransformColumnTypes(#"Pivot No Aggregation",{ {"ID", Int64.Type}, {"Task", type text}, {"city", type text}, {"Type of Business", type text}, {"Education need", type text}, {"Total", Int64.Type}, {"When", type text}, {"Degree", type text}}) in #"Changed Type"Results from your data:
- 8 months ago
Hi HaiVN , Thank you for reaching out to the Microsoft Community Forum.
I normalized the form output in Power Query so each education need becomes a separate row instead of duplicating the dataset. This makes the model scalable and suitable for analysis by education type, degree, and timing. The transformation uses unpivot/pivot logic and requires no manual maintenance if more education needs are added.
To help you better understand the implementation, I’ve attached the .pbix file for your reference. Please take a look at it and let me know your observations.
- 8 months ago
HI HaiVN just change the source with your actual table.
Here is complete m-code
let Source = Table.FromRows( { {1, "Project A", "Hanoi", "Manufacturing", "Data Science", 5, "Q1 2025", "Master", "AI/ML", 3, "Q2 2025", "PhD", "Cybersecurity", 2, "Q3 2025", "Bachelor"} }, type table [ ID = Int64.Type, Task = text, city = text, #"Type of Business" = text, #"Education need 1" = text, #"Total 1" = Int64.Type, #"When 1" = text, #"Degree 1" = text, #"Education need 2" = text, #"Total 2" = Int64.Type, #"When 2" = text, #"Degree 2" = text, #"Education need 3" = text, #"Total 3" = Int64.Type, #"When 3" = text, #"Degree 3" = text ] ), FixedColumns = {"ID", "Task", "city", "Type of Business"}, Unpivoted = Table.UnpivotOtherColumns(Source, FixedColumns, "Attribute", "Value"), AddBaseName = Table.AddColumn(Unpivoted, "BaseName", each Text.TrimEnd(Text.BeforeDelimiter([Attribute], " ", {0, RelativePosition.FromEnd}), {" "})), AddGroupNum = Table.AddColumn(AddBaseName, "GroupNum", each Text.AfterDelimiter([Attribute], " ", {0, RelativePosition.FromEnd})), Filtered = Table.SelectRows(AddGroupNum, each [Value] <> null and [Value] <> ""), Cleaned = Table.RemoveColumns(Filtered, {"Attribute"}), Pivoted = Table.Pivot(Cleaned, List.Distinct(Cleaned[BaseName]), "BaseName", "Value"), Final = Table.RemoveColumns(Pivoted, {"GroupNum"}) in FinalThanks
Hi HaiVN , Thank you for reaching out to the Microsoft Community Forum.
I normalized the form output in Power Query so each education need becomes a separate row instead of duplicating the dataset. This makes the model scalable and suitable for analysis by education type, degree, and timing. The transformation uses unpivot/pivot logic and requires no manual maintenance if more education needs are added.
To help you better understand the implementation, I’ve attached the .pbix file for your reference. Please take a look at it and let me know your observations.