Forum Discussion
asm495
3 years agoRegular Visitor
Aggregating multiple key-value columns
I have some data in this format: Id Question1 Value1 Question2 Value2 Question3 Value3 aaaa Pears 4 Apples 2 Oranges 5 aaab Oranges 2 Pears 5 Apples 7 aaac Kiwi 1 ...
- 3 years ago
Unpivot all columns except ID.
Select the Attribute column. Split it by delimiter (from Non-digit to digit).
Then Pivot the column Attribute.1 with Value in Values and choose Don't Aggregate in the advanced section.
- 3 years ago
asm495 Here is another way to solve this, a bit complex, but good to know how to do purely using M language.
let Source = Table.FromRows ( Json.Document ( Binary.Decompress ( Binary.FromText ( "i45WSgQCJR2lgNTEomIgbQLEjgUFOakgjhEQ+xcl5qWDeaZKsTpg9UkowkZIuk2RdZvD1CcDOd6Z5ZlAyhBFqwWycmOl2FgA", BinaryEncoding.Base64 ), Compression.Deflate ) ), let _t = ( ( type nullable text ) meta [ Serialized.Text = true ] ) in type table [ Id = _t, Question1 = _t, Value1 = _t, Question2 = _t, Value2 = _t, Question3 = _t, Value3 = _t ] ), ChangedType = Table.TransformColumnTypes ( Source, { { "Id", type text }, { "Question1", type text }, { "Value1", Int64.Type }, { "Question2", type text }, { "Value2", Int64.Type }, { "Question3", type text }, { "Value3", Int64.Type } } ), AddedCustom = Table.AddColumn ( ChangedType, "Custom", each let TargetCols = Record.RemoveFields ( _, "Id" ), RecToList = Record.ToList ( TargetCols ), Questions = List.Select ( Record.FieldNames ( TargetCols ), each Text.StartsWith ( _, "Question" ) ), SplitIntoGroups = List.Split ( RecToList, 2 ), ToRecords = List.Transform ( SplitIntoGroups, each Record.FromList ( _, { "Option", "Value" } ) ), Result = Table.FromColumns ( { Questions } & Table.ToColumns ( Table.FromRecords ( ToRecords ) ), type table [ Question = text, Option = text, Value = Int64.Type ] ) in Result, type table [ Question = text, Option = text, Value = Int64.Type ] ), RemovedOtherColumns = Table.SelectColumns ( AddedCustom, { "Id", "Custom" } ), ExpandedCustom = Table.ExpandTableColumn ( RemovedOtherColumns, "Custom", { "Question", "Option", "Value" }, { "Question", "Option", "Value" } ) in ExpandedCustom
AntrikshSharma
Community Champion
3 years agoasm495 Here is another way to solve this, a bit complex, but good to know how to do purely using M language.
let
Source = Table.FromRows (
Json.Document (
Binary.Decompress (
Binary.FromText (
"i45WSgQCJR2lgNTEomIgbQLEjgUFOakgjhEQ+xcl5qWDeaZKsTpg9UkowkZIuk2RdZvD1CcDOd6Z5ZlAyhBFqwWycmOl2FgA",
BinaryEncoding.Base64
),
Compression.Deflate
)
),
let
_t = ( ( type nullable text ) meta [ Serialized.Text = true ] )
in
type table [
Id = _t,
Question1 = _t,
Value1 = _t,
Question2 = _t,
Value2 = _t,
Question3 = _t,
Value3 = _t
]
),
ChangedType = Table.TransformColumnTypes (
Source,
{
{ "Id", type text },
{ "Question1", type text },
{ "Value1", Int64.Type },
{ "Question2", type text },
{ "Value2", Int64.Type },
{ "Question3", type text },
{ "Value3", Int64.Type }
}
),
AddedCustom = Table.AddColumn (
ChangedType,
"Custom",
each
let
TargetCols = Record.RemoveFields ( _, "Id" ),
RecToList = Record.ToList ( TargetCols ),
Questions =
List.Select (
Record.FieldNames ( TargetCols ),
each Text.StartsWith ( _, "Question" )
),
SplitIntoGroups = List.Split ( RecToList, 2 ),
ToRecords =
List.Transform (
SplitIntoGroups,
each Record.FromList ( _, { "Option", "Value" } )
),
Result =
Table.FromColumns (
{ Questions } & Table.ToColumns ( Table.FromRecords ( ToRecords ) ),
type table [ Question = text, Option = text, Value = Int64.Type ]
)
in
Result,
type table [ Question = text, Option = text, Value = Int64.Type ]
),
RemovedOtherColumns = Table.SelectColumns ( AddedCustom, { "Id", "Custom" } ),
ExpandedCustom =
Table.ExpandTableColumn (
RemovedOtherColumns,
"Custom",
{ "Question", "Option", "Value" },
{ "Question", "Option", "Value" }
)
in
ExpandedCustom
asm495
3 years agoRegular Visitor
I have tested this and this also works well. One thing that anyone else who uses this solution should be aware of is that it is required for the sequence of the original columns to be: Question,Value,Question,Value,...
My data wasn't in this sequence and I had to correct this before it would work.