topic update a pipleline variable from within a notebook using sql in Data Engineering

update a pipleline variable from within a notebook using sql

Mightyshax — Mon, 25 Nov 2024 16:56:00 GMT

hi i have a notebook SparkSQL and want to update a data pipeline paremeter with the number of rows that have been inserted by the notebook so i can check thsi before i move onto the next pipeline activity. Not sure how to actually update the parameter value using SparkSQL in the notebook?

Re: update a pipleline variable from within a notebook using sql

frithjof_v — Mon, 25 Nov 2024 23:46:21 GMT

See this thread:

https://community.fabric.microsoft.com/t5/Data-Pipeline/Referencing-notebook-exit-value-as-a-variable-in-a-data-pipeline/m-p/3507053

Ps. mssparkutils shall be replaced by notebookutils. https://learn.microsoft.com/en-us/fabric/data-engineering/notebook-utilities#exit-a-notebook