Forum Discussion
Ostrzak
Helper II
Helper IIReferencing notebook exit value as a variable in a data pipeline
Hi everyone, In general: I am trying to build a pipeline around multiple PySpark notebooks, with some additional pipeline features. To achieve it I need to pass values from a notebook to a pipeli...
For my own implementation the delimited string just wasnt flexible enough - so here's a version using a JSON object as the notebook return value:
Notebook pyspark:
lResult={"numUpdated":10,"message":"success","fruit":["apple","orange","grapes"]} mssparkutils.notebook.exit(str(lResult))Expression for varMessageStr:
Anonymous(activity('Test Notebook').output.result.exitValue).messageExpression for varNumUpdatedInt:
INT(json(activity('Test Notebook').output.result.exitValue).numUpdated)Expression for varFruitArray:
Anonymous(activity('Test Notebook').output.result.exitValue).fruit
Nero
Advocate II
Advocate IIFor my own implementation the delimited string just wasnt flexible enough - so here's a version using a JSON object as the notebook return value:
Notebook pyspark:
lResult={"numUpdated":10,"message":"success","fruit":["apple","orange","grapes"]}
mssparkutils.notebook.exit(str(lResult))
Expression for varMessageStr:
Anonymous(activity('Test Notebook').output.result.exitValue).message
Expression for varNumUpdatedInt:
INT(json(activity('Test Notebook').output.result.exitValue).numUpdated)
Expression for varFruitArray:
Anonymous(activity('Test Notebook').output.result.exitValue).fruit
Bravo! This was extremely helpful!
Many thanks!