Pipeline containing 5 notebooks. can it be parameterised

[DebbieE]

I have the following pipeline

They have to be run asynchrnously in this order

I wonder if there was a way to parameterise so you only need one notebook that runs for each item? 

I guess you would need the Workspace and Notebook in another file somewhere you can use? Is there any good documentation i can look at. I want to try doing it as simply as possible

[DebbieE]

I have cracked it. I added a csv of the Notebooks WorkspaceName, WorkspaceID, NotebookName, NotebookID. Then I did a lookup to the csv file.  Then a for each loop. Sequential. Items 

@activity('GetNotebookNames').output.value

In the for each loop added a Notebook

Workspace ID @item().workspaceID

 Notebook ID @item().notebookID

This seemed to do the trick

[AndyDDC]

if you really need separate notebooks, how about running the notebooks in parallel using runmultiple?

https://pl.seequality.net/run-multiple-notebooks-in-parallel-using-runmultiple-in-microsoft-fabric/

[DebbieE]

I don't want to run them in parallel. i want to run them in sequence. But I can see that there is a way of using a master notebook to run them with this. So I might be able to use that.

I was expecting a solution where I would have a notebook with each parameter needed to run each notebook. And iterate through that until it was completed

[Dronec]

You can easily do exactly that with the solution I described. For-each task that provides parameters to the notebook inside

[Dronec]

If you're processing them in the same way (i.e. no special logic for Dim Episode comparing to Dim Task), you generally don't need more than one notebook with parameters. Example of one of mine:

Notebook with parameters:

Pipeline task that sends parameters to this notebook:

[frithjof_v]

Any particular reason why you don't process all the tables inside one notebook?

[DebbieE]

Yes. To keep each notebook as simple as possible, Otherwise it would be really long. In the same way as I would process each as a stored procedure