Forum Discussion

kiara-mehra35's avatar
kiara-mehra35
Icon for Advocate II rankAdvocate II
18 days ago
Solved

Microsoft Fabric for GCC data operations: how would you structure the platform for scale?

A GCC implementation we examined involved more than just implementing a data platform; it required the realization of a scalable data engineering function to adapt to growing data volumes, expansion ...
  • ShivekMaharaj's avatar
    16 days ago

    Hi kiara-mehra35​,

    Yes, I would use a federated model rather than having one central Fabric team own every pipeline and dataset.

    Microsoft’s current Fabric deployment guidance treats workspaces as the main collaboration/security boundary, while domains can group those workspaces by business area and support delegated governance.

    I would split responsibilities roughly like this:

    Central Fabric/platform team:
    - capacity and tenant settings
    - workspace/domain provisioning
    - naming and security standards
    - Git / deployment pipeline patterns
    - shared monitoring and operational standards
    - reusable templates and common libraries
    
    Domain engineering teams:
    - ingestion pipelines
    - notebooks and transformations
    - domain-specific data-quality rules
    - ownership of bronze/silver/gold data products
    - business-specific SLAs and support

    For Dev/Test/Prod, I would keep the environments separated and use Git plus Fabric deployment pipelines for promotion rather than letting engineers modify Production directly.

    Microsoft’s CI/CD guidance also recommends treating orchestration and environment-specific configuration as part of the deployment design.

    For shared pipelines or data products, I would distinguish between shared platform capability and shared business logic.

    For example:

    common ingestion framework / logging / audit utilities
    -> centrally maintained
    
    Sales-specific ingestion and transformation rules
    -> owned by the Sales domain team
    
    trusted Customer or Product data product used by several domains
    -> one clearly assigned owning domain/team, with a published contract and SLA

    That avoids two extremes:

    central team becomes a bottleneck

    or

    every domain invents its own standards and duplicated pipelines

    I would therefore aim for centralized guardrails, decentralized ownership.

    Fabric’s medallion guidance also supports separating layers and responsibilities across workspaces when that improves governance and operational ownership.

    AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.