Forum Discussion

girishtharwani2's avatar
girishtharwani2
Icon for Helper III rankHelper III
1 day ago

Best orchestration choice to reduce Fabric capacity usage when transformations run in Snowflake

Hi,

I’m looking for advice on reducing Microsoft Fabric capacity usage. The same Fabric capacity is used for BI, so I want to avoid unnecessary data integration workloads affecting BI.

Our current flow is:

  1. Fabric Data Factory pipelines ingest data from external sources into ADLS.
  2. Fabric Data Factory pipelines load data from ADLS into Snowflake Silver.
  3. Fabric Data Factory pipelines trigger Snowflake Silver-to-Gold processing.

Most transformations run inside Snowflake. Fabric Data Factory is mainly handling ingestion, loading, and orchestration. These pipelines currently run on Fabric’s native schedules.

Would moving orchestration to a Fabric Apache Airflow Job reduce Fabric capacity consumption, or would it add more capacity usage while the Data Factory pipeline activities still run in Fabric? For this type of setup, what approach would you recommend to reduce Fabric capacity impact while keeping reliable scheduling and monitoring?

data engineering
data pipeline

1 Reply

  • Kagiyama_yutaka's avatar
    Kagiyama_yutaka
    Icon for Skilled Sharer rankSkilled Sharer
    21 hours ago

    I think moving just the scheduler to Fabric Airflow won't reduce CU usage. Pipeline charges remain, plus 5/10 CUs while the Airflow pool runs. Keep native schedules. Check DataMovement, ActivityRun and BI throttling in Capacity Metrics. Background CU usage is smoothed over 24 hours, so shifting run times alone won't isolate BI.

    If Copy dominates, move ingestion/loading to ADF or a separate Fabric capacity. Costs may shift or rise. Snowflake Tasks can handle Silver-to-Gold once loads complete. Monitor failures across systems.