Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
8 years ago
Solved

Load JSON array of values with field names in another element

Hi everyone,

 

I have JSON that contains an object representation of table of records.

The JSON object has two properties, "fields" and "results".

"fields" property contains array of objects defining the list of columns and their datatype

"results" property contains records, each consisting of an array of values corresponding to the columns in "fields" array

 

{
    "fields": [
        {
            "field": "field1",
            "type": "string"
        },
        {
            "field": "field2",
            "type": "string"
        },
        {
            "field": "field3",
            "type": "integer"
        },
        {
            "field": "eventTimestamp",
            "type": "date"
        }
    ],
    "results": [
        [
            "field1value1",
            "field2value1",
            10,
            "2018-05-31T22:55:31.187+0000"
        ],
        [
            "field1value2",
            "field2value2",
            20,
            "2018-05-31T22:55:31.187+0000"
        ],
        [
            "field1value3",
            "field2value3",
            30,
            "2018-05-31T22:55:31.187+0000"
        ]
    ]
}

 

My goal is to parse this data into dataset shaped like that with as the following table:

field1          field2          field3      eventTimestamp
field1value1    field2value1    10          2018-05-31T22:55:31.187+0000
field1value2    field2value2    20          2018-05-31T22:55:31.187+0000
field1value3    field2value3    30          2018-05-31T22:55:31.187+0000

I can get the "fields" table expanded without any problems:

let
    Source = Json.Document(File.Contents("C:\test\TestJSON.json")),
    fields = Source[fields],
    #"Converted to Table" = Table.FromList(fields, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandRecordColumn(#"Converted to Table", "Column1", {"label", "field", "type"}, {"label", "field", "type"})
in
    #"Expanded Column1"

 

 

 

However, I am not certain how to expand the array of values into table with the columns.

 

I can do Table.TransformColumns:

let
    Source = Json.Document(File.Contents("C:\test\TestJSON.json")),
    results = Source[results],
    #"Converted to Table" = Table.FromList(results, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Extracted Values" = Table.TransformColumns(#"Converted to Table", {"Column1", each Text.Combine(List.Transform(_, Text.From)), type text})
in
    #"Extracted Values"

... and it produces a single column with values joined together:

 

I can also do Table.ExpandListColumn:

let
    Source = Json.Document(File.Contents("C:\test\TestJSON.json")),
    results = Source[results],
    #"Converted to Table" = Table.FromList(results, Splitter.SplitByNothing(), null, null, ExtraValues.Error),
    #"Expanded Column1" = Table.ExpandListColumn(#"Converted to Table", "Column1")
in
    #"Expanded Column1"

but it produces multiple rows for single column, one for each record:

 

What I really want is to create multiple columns out of values.

 

If I was writing this in C#, I'd be create table with columns from "fields", and then looping through objects in "results" array, and load a table with columns by position.

 

My M is not very good just yet. How can I do this here?

 

Thank you!

Daniel

  • Anonymous's avatar
    Anonymous
    8 years ago

    Hi Anonymous,

     

    You can refer to below sample to convert source data.(I attach sample file below)


    Regards,

    Xiaoxin Sheng

2 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi Anonymous,

     

    You can refer to below sample to convert source data.(I attach sample file below)


    Regards,

    Xiaoxin Sheng

    • Anonymous's avatar
      Anonymous
      Not applicable

      Thank you very much.

       

       

      Very interesting about "each Table.Transpose" usage to create a table out of the array of "values".

       

      Another great new thing for me is using of List.Zip function to pass list of column names from "fields" as input for column names for "values".

       

      Finally, I like the custom function to analyze the data type of the value in "values" list to use as input for the column datatype.

       

      This helps a lot!

      Daniel