Forum Discussion
How to run multiple code files from a Folder in a Notebook
Hi nilendraFabric ,
Thanks for reaching out to the Microsoft fabric community forum.
If that is the requirement then the possible solution you could use is to use runmultiple() using DAG
NotebookUtils (former MSSparkUtils) for Fabric - Microsoft Fabric | Microsoft Learn
It may be painful to hardcode notebook paths for all the existing notebooks but this will get what you want i.e to run all the notebooks existing in one folder
Example taken from the mentioned document:# run multiple notebooks with parameters DAG = { "activities": [ { "name": "NotebookSimple", # activity name, must be unique "path": "NotebookSimple", # notebook path "timeoutPerCellInSeconds": 90, # max timeout for each cell, default to 90 seconds "args": {"p1": "changed value", "p2": 100}, # notebook parameters }, { "name": "NotebookSimple2", "path": "NotebookSimple2", "timeoutPerCellInSeconds": 120, "args": {"p1": "changed value 2", "p2": 200} }, { "name": "NotebookSimple2.2", "path": "NotebookSimple2", "timeoutPerCellInSeconds": 120, "args": {"p1": "changed value 3", "p2": 300}, "retry": 1, "retryIntervalInSeconds": 10, "dependencies": ["NotebookSimple"] # list of activity names that this activity depends on } ], "timeoutInSeconds": 43200, # max timeout for the entire DAG, default to 12 hours "concurrency": 50 # max number of notebooks to run concurrently, default to 50 } notebookutils.notebook.runMultiple(DAG, {"displayDAGViaGraphviz": False})
If you find this post helpful, please mark it as an "Accept as Solution" and consider giving a KUDOS.
Thanks and Regards
The requirement is that there are multiple module. And each module has different scripts.
existing setup is like Function module have all the functions and so on.
so customer is not looking to change the script name.
the idea is to place them the way they are in existing non Fabric system.
thanks spencer_sa n v-nmadadi-msft for the reply though
Hi nilendraFabric ,
Thanks for reaching out to the Microsoft fabric community forum.
If that is the requirement then the possible solution you could use is to use runmultiple() using DAG
NotebookUtils (former MSSparkUtils) for Fabric - Microsoft Fabric | Microsoft Learn
It may be painful to hardcode notebook paths for all the existing notebooks but this will get what you want i.e to run all the notebooks existing in one folder
Example taken from the mentioned document:# run multiple notebooks with parameters DAG = { "activities": [ { "name": "NotebookSimple", # activity name, must be unique "path": "NotebookSimple", # notebook path "timeoutPerCellInSeconds": 90, # max timeout for each cell, default to 90 seconds "args": {"p1": "changed value", "p2": 100}, # notebook parameters }, { "name": "NotebookSimple2", "path": "NotebookSimple2", "timeoutPerCellInSeconds": 120, "args": {"p1": "changed value 2", "p2": 200} }, { "name": "NotebookSimple2.2", "path": "NotebookSimple2", "timeoutPerCellInSeconds": 120, "args": {"p1": "changed value 3", "p2": 300}, "retry": 1, "retryIntervalInSeconds": 10, "dependencies": ["NotebookSimple"] # list of activity names that this activity depends on } ], "timeoutInSeconds": 43200, # max timeout for the entire DAG, default to 12 hours "concurrency": 50 # max number of notebooks to run concurrently, default to 50 } notebookutils.notebook.runMultiple(DAG, {"displayDAGViaGraphviz": False})
If you find this post helpful, please mark it as an "Accept as Solution" and consider giving a KUDOS.
Thanks and Regards