Forum Discussion

alexandremichi's avatar
alexandremichi
Regular Visitor
1 day ago
Solved

Sharing tooling across Fabric capacities

Hi all,

Following the recommended multi-workspace, multi-capacity pattern, we use separate workspaces for different ETL activities. Shared tooling and Environments live in a central workspace, which works well when everything is on one capacity.

With dev and prod on separate capacities, items can only attach Environments from workspaces on the same capacity. That means our dev items can’t use the Environment in our central tooling workspace.

What’s the recommended way to share tools across capacities without maintaining a separate Environment for each workspace? Is cross-capacity Environment sharing on the roadmap?

Thank you for your advice and recommendations.

  • Hi alexandremichi​,

    Echoing Murtaza_Ghafoor​: as far as I know, an Environment can only be attached to items on the same capacity, so the dependable pattern is one Environment per capacity, kept identical through source control and automation instead of by hand. I have not seen a published date for cross-capacity sharing, so the Fabric roadmap (release plan) is the place to confirm it. Below is how I would make "one per capacity" cheap to maintain.

    1. Treat the Environment as code

    The point is that the central tooling workspace stops being the source of truth, and a Git repository becomes it. Each capacity gets a copy of the same definition:

    Repository
      environments/
        etl-tooling/
          Libraries/         -> custom .whl files, public/custom library lists
          Setting/           -> Spark compute and runtime settings
          environment.yml    -> library definition
      notebooks/ pipelines/  -> items that reference the environment by name

    2. Deploy the same definition to every capacity

    Pick one promotion mechanism and use it for the Environment as well as for the other items, so Dev and Prod can never drift:

    Option A  Fabric Git integration + deployment pipelines -> promote Dev -> Test -> Prod stages
    Option B  fabric-cicd (Python) from Azure DevOps       -> publish items per target workspace
    Option C  REST API / Fabric CLI                         -> publish and update the environment from a pipeline step

    A sketch of option B, with the environment type included in the publish and the target workspace chosen per stage:

    from fabric_cicd import FabricWorkspace, publish_all_items
    
    target = FabricWorkspace(
        workspace_id="<dev-or-prod-workspace-id>",
        environment="DEV",                       # parameter.yml selects stage values
        repository_directory="./",
        item_type_in_scope=["Environment", "Notebook", "DataPipeline"],
    )
    publish_all_items(target)

    3. Keep what differs by stage out of the definition

    Same in every stage  -> libraries, Spark settings, runtime version
    Different per stage  -> connection details, lakehouse names, workspace IDs (Variable Library or parameter file)
    Versioning           -> pin library versions and tag the repo so Dev and Prod can be compared quickly

    4. Options if you want to avoid duplicates entirely

    • Publish shared Python code as a versioned wheel in one artifact feed or storage location and have each Environment reference the same wheel version, so there is one build and many consumers.
    • For small helper code, inline installs in the notebook (with pinned versions) avoid an Environment at the cost of slower session startup.
    • If sharing across workspaces is the real goal rather than sharing across capacities, keeping dev and prod on one capacity is the simplest option, at the price of the isolation you gain from separate capacities.

    So my overall approach would be:

    Source of truth -> Git repo with the Environment definition
    Deploy          -> same definition published to each capacity by CI/CD
    Stage values    -> Variable Library / parameters, never baked into the Environment
    Roadmap         -> confirm cross-capacity Environment sharing in the release plan

    AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.

3 Replies

  • Hi both,

    Thank you for the feedback and insights, they are much appreciated!

    Our team came up with fairly similar ideas, not as detailed though. Ideally we would be able to share both across capacities and workspaces to have a truly centralised tooling, but I agree that a dedicated environment per capacity seems to be the better option given the current constraints. Just feels weird to have a production environment hosted on a dev capacity 😅

  • Hi alexandremichi​,

    Echoing Murtaza_Ghafoor​: as far as I know, an Environment can only be attached to items on the same capacity, so the dependable pattern is one Environment per capacity, kept identical through source control and automation instead of by hand. I have not seen a published date for cross-capacity sharing, so the Fabric roadmap (release plan) is the place to confirm it. Below is how I would make "one per capacity" cheap to maintain.

    1. Treat the Environment as code

    The point is that the central tooling workspace stops being the source of truth, and a Git repository becomes it. Each capacity gets a copy of the same definition:

    Repository
      environments/
        etl-tooling/
          Libraries/         -> custom .whl files, public/custom library lists
          Setting/           -> Spark compute and runtime settings
          environment.yml    -> library definition
      notebooks/ pipelines/  -> items that reference the environment by name

    2. Deploy the same definition to every capacity

    Pick one promotion mechanism and use it for the Environment as well as for the other items, so Dev and Prod can never drift:

    Option A  Fabric Git integration + deployment pipelines -> promote Dev -> Test -> Prod stages
    Option B  fabric-cicd (Python) from Azure DevOps       -> publish items per target workspace
    Option C  REST API / Fabric CLI                         -> publish and update the environment from a pipeline step

    A sketch of option B, with the environment type included in the publish and the target workspace chosen per stage:

    from fabric_cicd import FabricWorkspace, publish_all_items
    
    target = FabricWorkspace(
        workspace_id="<dev-or-prod-workspace-id>",
        environment="DEV",                       # parameter.yml selects stage values
        repository_directory="./",
        item_type_in_scope=["Environment", "Notebook", "DataPipeline"],
    )
    publish_all_items(target)

    3. Keep what differs by stage out of the definition

    Same in every stage  -> libraries, Spark settings, runtime version
    Different per stage  -> connection details, lakehouse names, workspace IDs (Variable Library or parameter file)
    Versioning           -> pin library versions and tag the repo so Dev and Prod can be compared quickly

    4. Options if you want to avoid duplicates entirely

    • Publish shared Python code as a versioned wheel in one artifact feed or storage location and have each Environment reference the same wheel version, so there is one build and many consumers.
    • For small helper code, inline installs in the notebook (with pinned versions) avoid an Environment at the cost of slower session startup.
    • If sharing across workspaces is the real goal rather than sharing across capacities, keeping dev and prod on one capacity is the simplest option, at the price of the isolation you gain from separate capacities.

    So my overall approach would be:

    Source of truth -> Git repo with the Environment definition
    Deploy          -> same definition published to each capacity by CI/CD
    Stage values    -> Variable Library / parameters, never baked into the Environment
    Roadmap         -> confirm cross-capacity Environment sharing in the release plan

    AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.

  • Hi alexandremichi​,
    In Microsoft Fabric, Environments are currently capacity-bound, so items generally cannot attach an Environment hosted on a different capacity.

    Recommended approach:

    • Maintain a separate Environment on each capacity (Dev/Prod).
    • Keep Environment definitions, libraries, and configuration centrally version-controlled in Azure DevOps.
    • Use CI/CD to deploy and synchronize Environment configurations across capacities.

    This avoids manually maintaining different configurations while respecting capacity boundaries.

    Cross-capacity Environment sharing would be a useful enhancement, but it should be properly checked with fabric road map.

    If this helps, ✓ Mark as Kudos | Help Other