topic Flattening nested JSON values in a dataflow with varying keys. in Dataflow

Flattening nested JSON values in a dataflow with varying keys.

Velegandla — Thu, 06 Jun 2024 02:39:39 GMT

We are using Azrue DevOps REST API calls to return JSON files and storing them in blob. Then we perform a dataflow to transform the data. The issue is a portion of the JSON being stored in blob has varying keys.

When we specify the columns to map in a Select action, we are selecting specifically one of the varying keys from a list of options. But need to map ALL of these – we cannot manually specify these because the data source is so large.

We cannot implement a standard name for this section of the JSON. A wildcard for { } would work ideally but is not supported. We do not care what the keys are, just the contents (id, name).

Select Action:
Source Column:
resources.pipelines.{src-release}.pipeline.id
resources.pipelines.{src-release}.pipeline.name

resources.pipelines.{build }.pipeline.id

resources.pipelines.{build }.pipeline.name

Mapping Name as:
‘pipelineID’

‘pipelineName’

Below is a JSON snippet which highlights the key from the source JSON

Example of Select action mapping – each key shows as its own dropdown:

Re: Flattening nested JSON values in a dataflow with varying keys.

Anonymous — Thu, 06 Jun 2024 17:25:27 GMT

Hi Velegandla
Thanks for using Fabric Community.
Can you please confirm if you are trying the above scenario with ADF dataflow or Fabric Dataflows?
thanks

Re: Flattening nested JSON values in a dataflow with varying keys.

Velegandla — Fri, 07 Jun 2024 01:29:08 GMT

Anonymous

ADF dataflow

Re: Flattening nested JSON values in a dataflow with varying keys.

Anonymous — Mon, 10 Jun 2024 04:51:53 GMT

Hi Velegandla

Apologies for the inconvenience, this community is specifically for discussions and questions related to Microsoft Fabric. For queries related to Azure Data Factory, we recommend using the dedicated Microsoft Q&A forum where they are ready to assist you better. Here's the link to post your query in Microsoft Q&A : Ask a question - Microsoft Q&A.

Thanks.