Forum Discussion

DebbieE's avatar
DebbieE
Icon for Community Champion rankCommunity Champion
2 years ago

Pipeline containing 5 notebooks. can it be parameterised

I have the following pipeline   They have to be run asynchrnously in this order   I wonder if there was a way to parameterise so you only need one notebook that runs for each item?    I g...
data engineering
data pipeline
DebbieE's avatar
DebbieE
Icon for Community Champion rankCommunity Champion
2 years ago

I don't want to run them in parallel. i want to run them in sequence. But I can see that there is a way of using a master notebook to run them with this. So I might be able to use that.

 

I was expecting a solution where I would have a notebook with each parameter needed to run each notebook. And iterate through that until it was completed

Dronec's avatar
Dronec
Icon for Advocate II rankAdvocate II
2 years ago

You can easily do exactly that with the solution I described. For-each task that provides parameters to the notebook inside