Fabric capacity issue - Python notebooks

In this scenario, we can be sure that each notebook requires its own session for running. A python notebook runs on a 2 VCore, 16 GB RAM single node cluster. Having three of these running around the clock can consume a good chunk of your capacity. 

I think it's worth it to reconsider the orchestration with the following things in mind: 

- Running all three notebook from a single pipeline with High Concurrency for pipelines enabled in the Spark settings will cause all three notebook being run in the same session, limiting the amount of blocked VCores to what is consumed during a single session

- Alternatively, you can create a fourth ("control") notebook which leverages the runMultiple() function of notebookutils library to run all three notebooks within the same session, orchestrated from a notebook. The control notebook is then triggered from a data pipeline

Both of these option would result in your session being handed from one notebook to another rather than having all three notebook use their own. Implementing one of these options could help reduce the bound resources of your capacity. 

Hope this helps! 🙂