Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The Power BI Data Visualization World Championships is back! It's time to submit your entry. Live now!

Ability to parse JSON data from a notebook to Multiple Variables / Array

One of the more common coding tasks that I reuse often is using a Notebook to retrieve settings and information related to the workspace/environment/etc, to facilitate dynamic and meta-driven development. I will usually use an initial notebook that I run as the first step in a Pipeline which setups logging, retrieves lakehouse/workspace information, etc. I then exit the notebook returning a JSON string with the values gathered:

ex.

# All of these would have been pre-populated earlier in the code as needed....
ret_cfg = { 
    "workspace_id": workspace_id, 
    "workspace_name": workspace_name,
    "environment": environment,
    "metadata_lakehouse_id": lh_metadata_id,
    "bronze_lakehouse_id": lh_bronze_id
}
# Exit notebook with configuration
mssparkutils.notebook.exit(json.dumps(ret_cfg))

 

In the calling orchestration pipeline, I would then have to create individual Variables as needed, and set the values from the from the Output of the for each:

Shredda_0-1765910954479.png

 

However, as the return values increase than this causes the Pipeline to become cluttered and unmanageable.  I'm proposing a JSON variable or Key/Value Variable type (or something similar) in addition to the String/Array/Boolean/Integer types, and a way to parse and/or read the JSON value from the notebook (which could also be useful in other situations different to this.)

 

Status: New