Forum Discussion

adamcodes716's avatar
adamcodes716
Regular Visitor
2 years ago
Solved

Trying to expand a json array into multiple columns

I am trying to set up an ingestion from an event hub to a KQL database.  The difficulty in doing this is that there is a payload field that is a few levels down.  I am trying to do this via the UI using an Event Processor to do this and I can go most of the way there.  I can get to the point where I have a column with the payload:

 

Here is a simple example of that column:  [{"name":"itemA", "section":"34", "key":"1234"}]

 

I could use "expand" here, but this breaks out the payload into 3 rows.  The problem with this is that the "key" only exists in one of the rows and I have no way to tie it back to the other entries.  Is there another option using Event Processor?  I am trying to avoid doing this in code.

4 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi adamcodes716,

    Thanks for using Fabric Community.
    You will need to do this KQL Queryset with 2 lines of code: mv-expand operator and bag_unpack() in KQL language. You can put these inside a function and call this function from Update Policies to transform this data on the fly.

    Docs to refer -

    mv-expand operator - Azure Data Explorer & Real-Time Analytics | Microsoft Learn
    bag_unpack plugin - Azure Data Explorer & Real-Time Analytics | Microsoft Learn

    Hope this is helpful. Please let me know incase of further queries.

    • Anonymous's avatar
      Anonymous
      Not applicable

      Hi adamcodes716 ,

      We haven’t heard from you on the last response and was just checking back to see if your query was answered.
      Otherwise, will respond back with the more details and we will try to help .

      Thanks

      • Anonymous's avatar
        Anonymous
        Not applicable

        Hi adamcodes716 ,

        We haven’t heard from you on the last response and was just checking back to see if your query was answered.
        Otherwise, will respond back with the more details and we will try to help .

        Thanks