Forum Discussion

romovaro's avatar
romovaro
Icon for Responsive Resident rankResponsive Resident
26 days ago
Solved

Transform data.

HI

I have an issue with Transform data

I have an excel with different columns (attached)

and at the end I have 2 Column providing a code with a lot of information. (see example attached)

CEXTRA PARAM1  

PAYLOAD

I would like to have a different column for every parameter for analysis.

Tried Transform Data and pivot table but getting errors.

Example first 2 rows:

 
 

5 Replies

  • Hi romovaro​ - Please find the attached PBIX file with the required transformations applied. Both CEXTRA PARAM1 and PAYLOAD have been expanded into separate columns for analysis.

    Hope this helps!

     

     

  • Hi,

    In another tab of that Excel file, please show the expected result.

  • jubinsoni's avatar
    jubinsoni
    Icon for Memorable Member rankMemorable Member

    Hi romovaro​,

    To complement the PBIX shared above, here is the pattern written out, so you can reuse it on your real file and see why Pivot was failing.

    Pivot errors because the values in CEXTRA PARAM1 / PAYLOAD are one JSON text blob per row, not key/value rows. Parse each blob into a record first, then expand the record. Power Query then creates one column per parameter for you.

    let
        Source   = Excel.Workbook(File.Contents("C:\path\Book2.xlsx"), null, true),
        Data     = Source{[Item = "Sheet1", Kind = "Sheet"]}[Data],
        Promoted = Table.PromoteHeaders(Data, [PromoteAllScalars = true]),
    
        // parse both JSON columns; bad/empty rows become null instead of erroring
        Parsed = Table.TransformColumns(Promoted, {
            {"CEXTRA PARAM1", each try Json.Document(_) otherwise null},
            {"PAYLOAD",       each try Json.Document(_) otherwise null}}),
    
        // collect every key that appears in any row (not just the first one)
        KeysParam   = List.Distinct(List.Combine(List.Transform(
                        List.RemoveNulls(Parsed[#"CEXTRA PARAM1"]), Record.FieldNames))),
        KeysPayload = List.Distinct(List.Combine(List.Transform(
                        List.RemoveNulls(Parsed[PAYLOAD]), Record.FieldNames))),
    
        ExpParam   = Table.ExpandRecordColumn(Parsed, "CEXTRA PARAM1",
                        KeysParam, List.Transform(KeysParam, each "PARAM." & _)),
        ExpPayload = Table.ExpandRecordColumn(ExpParam, "PAYLOAD",
                        KeysPayload, List.Transform(KeysPayload, each "PAYLOAD." & _))
    in
        ExpPayload

    Two details that matter on real data:

    collecting keys from ALL rows -> new parameters appear as columns on refresh
      (the UI expand button hard-codes only the keys it saw in the preview)
    prefixes "PARAM." / "PAYLOAD." -> no clashes when both blobs share a key name
    nested objects/arrays -> those columns show Record/List
      -> expand them again the same way, or Text.Combine a list into one cell

    If the values look like key=value;key=value rather than JSON, swap Json.Document for a small splitter:

    each Record.FromList(
        List.Transform(Text.Split(_, ";"), (kv) => Text.AfterDelimiter(kv, "=")),
        List.Transform(Text.Split(_, ";"), (kv) => Text.BeforeDelimiter(kv, "=")))

    So my overall approach would be:

    Json.Document (or key=value splitter) -> record per row
    -> gather keys across all rows -> Table.ExpandRecordColumn
    -> prefix column names -> set data types -> analyze

    AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi romovaro​ 

    We would like to inquire whether have you got the chance to check the solutions provided by other users in community to resolve the issue. We hope the information provided helps to clear the query. Should you have any further queries, kindly feel free to contact the Microsoft Fabric community.

  • v-csrikanth's avatar
    v-csrikanth
    Icon for Community Support rankCommunity Support

    Hi romovaro​ 

    We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.

    Thank you.