Forum Discussion

Ostrzak's avatar
Ostrzak
Icon for Helper II rankHelper II
2 years ago
Solved

Referencing notebook exit value as a variable in a data pipeline

Hi everyone,   In general: I am trying to build a pipeline around multiple PySpark notebooks, with some additional pipeline features. To achieve it I need to pass values from a notebook to a pipeli...
Need Help
pipelines
  • Nero's avatar
    Nero
    2 years ago

    For my own implementation the delimited string just wasnt flexible enough - so here's a version using a JSON object as the notebook return value: 

     

    Notebook pyspark:

     

    lResult={"numUpdated":10,"message":"success","fruit":["apple","orange","grapes"]}
mssparkutils.notebook.exit(str(lResult))

     

    Expression for varMessageStr:

     

    Anonymous(activity('Test Notebook').output.result.exitValue).message

     

    Expression for varNumUpdatedInt:

     

    INT(json(activity('Test Notebook').output.result.exitValue).numUpdated)

     

    Expression for varFruitArray:

     

    Anonymous(activity('Test Notebook').output.result.exitValue).fruit

     

Ostrzak's avatar
Ostrzak
Icon for Helper II rankHelper II
2 years ago

An update: I found out that changing Set variable activity to:

@activity('Notebook_name').output.result.exitValue

... solves the main problem.

 

I still haven't figured out how to pass multiple notebook values to pipeline variables, so if anyone could help me I will be grateful.

v-cboorla-msft's avatar
v-cboorla-msft
Icon for Microsoft Employee rankMicrosoft Employee
2 years ago

Hi Ostrzak 

 

Thanks for using Fabric Community.

We are reaching out to the internal team to get more information related to your query and will get back to you as soon as we have an update.