Fabric Notebook ExitValue how to select specific values

[Ka13]

Hi, 

I had a question in the Fabric Notebook.

In Fabric Notebook , From the ExitValue below output in Notebook , how to select specific value , like only schema and table and need to select values of schema and table output in Notebook.

ExitValue: {"parent1": {"name": "test", "Details": {"status": "active", "schema": "dbo","table" : "employee}}}

[v-prasare]

Hi @RajK2,

thanks for explaining the scenario. To better understand the issue and provide an accurate solution, could you please share a small set of sample data (with any sensitive information removed) along with the expected output? This will help ensure the guidance is aligned with your exact requirement.

Thanks,

Prashanth

MS fabric community support

[Ka13]

Hi Prasare , I tried the below solution provided by Pallavi it worked

In Notebook

import json
from notebookutils import notebook


nb_result = {
  "select_query" = select_query
}


notebook.exit(json.dumps(nb_result))

Then in pipeline, added notebook activity. In the dynamic content added the json

@Anonymous(activity('Notebook1).output.exitValue).select_query  

[v-prasare]

Hi @Ka13,

You can pass the formed SELECT query to the next Copy Activity by exiting the notebook with the query as JSON and referencing it using dynamic content in the Copy Activity source. This method works reliably in Fabric pipelines and is the recommended way to execute dynamic SQL from a notebook

Thanks,

prashanth

MS fabric community support

[Ka13]

Thanks Prasare, will test and let you know if any issues.

[Ka13]

@pallavi_r - Can you please help and suggest.

[pallavi_r]

Hi @Ka13 ,

The exitvalue returned from mssparkutils.notebook.run() is always a string. We need to first parse into json using function - json.loads to access its element. Please see the below code.

import json

parsed = json.loads(exit_value)

schema = parsed["parent1"]["Details"]["schema"]

table = parsed["parent1"]["Details"]["table"]

The above code worked for me. Please see and let me know if it resolves this issue.

If this post helps, please accept this as a solution. Appreciate your kudos.

Thanks,

Pallavi_r

[Ka13]

Thanks Pallavi for your reply. 

Can we pass the above ExitValue from one notebook to another Notebook activity and access the values of schema and table in another Notebook?. Can you please suggest 

[pallavi_r]

Hi @Ka13 

Yes definitely. Please see this. Below code is written in 2nd notebook calling another notebook.

exit_value = mssparkutils.notebook.run(

    "nb_begin",

    timeout_seconds=60

)

If this post helps, please accept this as a solution. Appreciate your kudos.

Thanks,

Pallavi

[Ka13]

Thanks Pallavi, in Details if we have one more field of columns like - id,name,city,country. And to extract columns.

can you please suggest.

[pallavi_r]

Hi @Ka13 ,

So here we need frame a dynamic sql. in the previous one we had parsed the string into json. Now we have to retrieve all the column names into string literal in a python list.

details = parsed["parent1"]["Details"]

schema = details["schema"]

table = details["table"]

columns = details["columns"]

getcol = [

    f"CAST({col} AS STRING) AS {col}"

    for col in columns

]

Then to create a dynamic sql expression, we need to merge all the elements from the above list into a single comma separated string using ",".join(column) 

select_query = f"""

SELECT

    {", ".join(getcol)}

FROM {schema}.{table}

"""

print(select_query)

working code below:

If this post helps, please accept this as a solution. Appreciate your kudos.

Thanks,

Pallavi

[Ka13]

Hi Pallavi, can we pass this formed select query in next Copy Activity?

[Ka13]

Thanks Pallavi, can we pass this select query formed to the next activity ( Copy Activity) ? , to execute the query. Will test the above and let you know if any questions.

[Ka13]

Hi Pallavi, can we pass this formed select query in next Copy Activity?

[pallavi_r · Accepted solution]

Hi @Ka13 ,

Yes we can read this query in a copy activity. First convert this select query to json 

import json

from notebookutils import notebook

nb_result = {

    "select_query": select_query

}

notebook.exit(json.dumps(nb_result))

Then in pipeline add notebook activity and connect to copy data. In the dynamic content of copy data, add this json expression.

@json(activity('Notebook1').output.exitValue).select_query

This works.

If this post helps, please accept this as a solution. Appreciate your kudos.

Thanks,

Pallavi_r

[Ka13]

Thanks Pallavi tested above it's working 

[Ka13]

Hi Pallavi,

in the getcol , the column names are getting split as below . Can you please suggest 

ID - AS I , D AS D

[pallavi_r]

Hi @Ka13 

Sorry for late reply. Can you please send me the exit value and parsed. 

exit_value = mssparkutils.notebook.run(

    "nb_first",

    timeout_seconds=60

)

exit_value

and this one

import json

parsed = json.loads(exit_value)

parsed

Thanks,

Pallavi