Shared data, one consistent view across teams.

hello!

This is a phenomenal real-world scenario. Healthcare analytics perfectly highlights the exact pain points that Microsoft Fabric was built to solve-specifically the nightmare of reconciling clinical compliance, operational metrics, and management dashboards over fragmented data.

In our architectural blueprints, we heavily lean toward a Hybrid Medallion Architecture (Combination based on the workload) using both the Lakehouse and the Warehouse layers. Here is a breakdown of how we structure this progression before it hits the Power BI Direct Lake reporting layer:

1. The Ingestion & Bronze Layer (Lakehouse First)

For data coming from EMR/EHR systems, operational tracking, and messy SharePoint files, we always ingest into a Lakehouse first. We leverage OneLake Shortcuts to connect to external operational file stores instantly without duplicating the raw data. This acts as our Bronze tier-storing raw Delta Parquet files securely.

2. The Transformation & Silver Layer (Spark Notebooks)

Healthcare data requires rigorous data quality, schema enforcement, and absolute compliance handling. We run Synapse Spark Notebooks (PySpark) directly over the Lakehouse tables to clean strings, handle null values, mask PII data to maintain HIPAA compliance, and structure the records.

3. The Serving & Gold Layer (The Lakehouse vs. Warehouse Decision)

This is where the workload split happens based on the audience you mentioned:

• For Clinical Teams and Data Scientists: We keep the Gold tables inside the Lakehouse. This allows data scientists to build predictive AI models directly over highly optimized Delta tables using PySpark without any ingestion lag.

• For Operations and Management Teams: We expose these Gold tables through a Fabric Data Warehouse. The Warehouse engine provides native SQL capabilities, cross-database JOINs, row-level security (RLS), and object-level permissions that enterprise database administrators and business intelligence developers expect.

The Magic Link: Power BI Direct Lake Mode

Regardless of whether the data sits in the Lakehouse or Warehouse compute tier, it all lands in the unified open-source Delta Parquet format inside OneLake. This allows our Power BI dashboards to query millions of rows of patient and operational records instantly via Direct Lake mode-completely bypassing legacy import refresh bottlenecks and ensuring everyone looks at the exact same truth.

For a multi-stakeholder domain like healthcare, a combination framework is definitely the gold standard! Curious to hear if your current setup at datatobiz is leaning more toward native Spark pipelines or T-SQL orchestration for your Silver-to-Gold transformations?