Forum Discussion
Ostrzak
Helper II
Helper IIReferencing notebook exit value as a variable in a data pipeline
Hi everyone, In general: I am trying to build a pipeline around multiple PySpark notebooks, with some additional pipeline features. To achieve it I need to pass values from a notebook to a pipeli...
For my own implementation the delimited string just wasnt flexible enough - so here's a version using a JSON object as the notebook return value:
Notebook pyspark:
lResult={"numUpdated":10,"message":"success","fruit":["apple","orange","grapes"]} mssparkutils.notebook.exit(str(lResult))Expression for varMessageStr:
Anonymous(activity('Test Notebook').output.result.exitValue).messageExpression for varNumUpdatedInt:
INT(json(activity('Test Notebook').output.result.exitValue).numUpdated)Expression for varFruitArray:
Anonymous(activity('Test Notebook').output.result.exitValue).fruit
Nero
Advocate II
Advocate IIFor my own implementation the delimited string just wasnt flexible enough - so here's a version using a JSON object as the notebook return value:
Notebook pyspark:
lResult={"numUpdated":10,"message":"success","fruit":["apple","orange","grapes"]}
mssparkutils.notebook.exit(str(lResult))
Expression for varMessageStr:
Anonymous(activity('Test Notebook').output.result.exitValue).message
Expression for varNumUpdatedInt:
INT(json(activity('Test Notebook').output.result.exitValue).numUpdated)
Expression for varFruitArray:
Anonymous(activity('Test Notebook').output.result.exitValue).fruit
Thank you so much for this solution - curious question for you: I was able to get it working only with
@json(activity('Test Notebook).output.status.Output.result.exitValue).param1
rather than
@json(activity('Test Notebook').output.result.exitValue).year
Do you know why this could be?
Do you know why this could be?
Apologies, the bottom should read: @json(activity('Test Notebook').output.result.exitValue).param1