Forum Discussion
Transform data.
HI
I have an issue with Transform data
I have an excel with different columns (attached)
and at the end I have 2 Column providing a code with a lot of information. (see example attached)
CEXTRA PARAM1
PAYLOAD
I would like to have a different column for every parameter for analysis.
Tried Transform Data and pivot table but getting errors.
Example first 2 rows:
Hi romovaro - Please find the attached PBIX file with the required transformations applied. Both CEXTRA PARAM1 and PAYLOAD have been expanded into separate columns for analysis.
Hope this helps!
5 Replies
- rajendraongole1
Super User
Hi romovaro - Please find the attached PBIX file with the required transformations applied. Both CEXTRA PARAM1 and PAYLOAD have been expanded into separate columns for analysis.
Hope this helps!
- Ashish_Mathur
Super User
Hi,
In another tab of that Excel file, please show the expected result.
- jubinsoni
Memorable Member
Hi romovaro,
To complement the PBIX shared above, here is the pattern written out, so you can reuse it on your real file and see why Pivot was failing.
Pivot errors because the values in CEXTRA PARAM1 / PAYLOAD are one JSON text blob per row, not key/value rows. Parse each blob into a record first, then expand the record. Power Query then creates one column per parameter for you.
let Source = Excel.Workbook(File.Contents("C:\path\Book2.xlsx"), null, true), Data = Source{[Item = "Sheet1", Kind = "Sheet"]}[Data], Promoted = Table.PromoteHeaders(Data, [PromoteAllScalars = true]), // parse both JSON columns; bad/empty rows become null instead of erroring Parsed = Table.TransformColumns(Promoted, { {"CEXTRA PARAM1", each try Json.Document(_) otherwise null}, {"PAYLOAD", each try Json.Document(_) otherwise null}}), // collect every key that appears in any row (not just the first one) KeysParam = List.Distinct(List.Combine(List.Transform( List.RemoveNulls(Parsed[#"CEXTRA PARAM1"]), Record.FieldNames))), KeysPayload = List.Distinct(List.Combine(List.Transform( List.RemoveNulls(Parsed[PAYLOAD]), Record.FieldNames))), ExpParam = Table.ExpandRecordColumn(Parsed, "CEXTRA PARAM1", KeysParam, List.Transform(KeysParam, each "PARAM." & _)), ExpPayload = Table.ExpandRecordColumn(ExpParam, "PAYLOAD", KeysPayload, List.Transform(KeysPayload, each "PAYLOAD." & _)) in ExpPayloadTwo details that matter on real data:
collecting keys from ALL rows -> new parameters appear as columns on refresh (the UI expand button hard-codes only the keys it saw in the preview) prefixes "PARAM." / "PAYLOAD." -> no clashes when both blobs share a key name nested objects/arrays -> those columns show Record/List -> expand them again the same way, or Text.Combine a list into one cellIf the values look like key=value;key=value rather than JSON, swap Json.Document for a small splitter:
each Record.FromList( List.Transform(Text.Split(_, ";"), (kv) => Text.AfterDelimiter(kv, "=")), List.Transform(Text.Split(_, ";"), (kv) => Text.BeforeDelimiter(kv, "=")))So my overall approach would be:
Json.Document (or key=value splitter) -> record per row -> gather keys across all rows -> Table.ExpandRecordColumn -> prefix column names -> set data types -> analyzeAI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.
- v-csrikanth
Community Support
Hi romovaro
We would like to inquire whether have you got the chance to check the solutions provided by other users in community to resolve the issue. We hope the information provided helps to clear the query. Should you have any further queries, kindly feel free to contact the Microsoft Fabric community.
- v-csrikanth
Community Support
Hi romovaro
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.
Thank you.