Forum Discussion

Sureshmannem's avatar
Sureshmannem
Frequent Visitor
1 year ago
Solved

Support - How to pass pipeline parameter values to Notebook (pyspark(python))

Hi Team,

 

We have a requirement to use pipeline parameters into notebook at the end of the pipelien execution.

I tried passing parameters using below approaches, it is not passing the values and resulting to pipeline failure. kindly suggest any working solution to pass pipeline parameters into Notebook (pyspark (python))

#1

file_category = mssparkutils.env.get_parameter("file_category")

copy_counts = mssparkutils.env.get_parameter("copy_counts")

#2
file_category = os.environ.get("file_category")
copy_counts = os.environ.get("copy_counts")
 
Thanks in advance
data engineering
Data Pipeline
  • Gpop13's avatar
    Gpop13
    1 year ago

    hi Sureshmannem ,

    Can you not set the base parameters of the notebooks with the pipeline variables something like below

     

    and then in the notebook you can directly use those variables

     

2 Replies

  • Gpop13's avatar
    Gpop13
    Advocate IV
    1 year ago

    hi Sureshmannem ,

    Can you not set the base parameters of the notebooks with the pipeline variables something like below

     

    and then in the notebook you can directly use those variables

     

    • Sureshmannem's avatar
      Sureshmannem
      Frequent Visitor
      1 year ago

      Hi Gpop13 

       

      Thanks for your prompt support!
      I realized I was overengineering the solution. I’ve now used the parameters directly in the notebook, and it works as expected.