Forum Discussion

AdarshChekodu's avatar
AdarshChekodu
Advocate IV
3 months ago
Solved

Spark Pool Usage Across Workspaces with Promoted Fabric Environments

Hi everyone, We have a centralized workspace in Microsoft Fabric where we manage and promote Environment items (e.g., Spark configurations). Multiple other workspaces and notebooks connect to these...
  • v-nmadadi-msft's avatar
    3 months ago

    Hi AdarshChekodu   ,
    Thanks for reaching out to the Microsoft Fabric Community forum. 

     

    In Microsoft Fabric, an Environment item is primarily a configuration and dependency management layer, not a shared Spark compute pool. Even when multiple notebooks across different workspaces reference the same centralized Environment, the actual Spark compute is still provisioned at the notebook/session level.

    This means that:

    • The shared Environment helps standardize settings such as:
      • Spark configurations
      • Libraries/packages
      • Runtime dependencies
    • However, it does not cause notebooks to share the same active Spark session or cluster instance.

    In practice, each notebook execution typically starts or attaches to its own Spark session/runtime context, even if the notebooks are using the exact same Environment item. So the compute is generally isolated per notebook/session, while the Environment acts as a reusable configuration template across workspaces.

     

    If your goal is to have multiple notebooks use the same spark session, there is a feature called high concurrency mode, you could attach multiple notebooks to the high concurrency session getting a spark session instantly to run the queries and achieve a greater session utilization rate.
    Configure high concurrency mode for notebooks - Microsoft Fabric | Microsoft Learn

    I hope this information helps. Please do let us know if you have any further queries.
    Thank you