Refresh duration of a semantic model plugged on a dataflow

BrunoM  Why the Refresh Might Be Slow

Here are some key considerations:

1. Dataflow Gen 1 Performance

• Gen 1 dataflows store data in CSV format on ADLS Gen2, which is less optimized for high-speed ingestion compared to Gen 2 (which supports Lakehouse, Warehouse, etc.).
• Reading large CSVs (20M rows per table) can be I/O intensive, especially if partitioning isn't used.

2. Semantic Model Import Mode

• Even without transformations, Power BI still loads data into VertiPaq memory engine, compresses it, and builds internal structures.
• A 1.6 GB model with 40M rows total can take time to process, especially if the dataflow refresh and semantic model refresh are not well synchronized.

3. No Incremental Refresh

• If you're doing a full refresh every time, Power BI re-ingests all 40M rows, which is costly.
• Consider enabling incremental refresh if your data supports it (e.g., has a date column).

4. Network and Capacity Load

• Even in Premium, performance can vary based on concurrent workloads, network latency, and backend throttling.
• Check the refresh history for signs of bottlenecks or retry attempts.
  
  

What You Can Do

• Switch to Dataflow Gen 2: It’s more performant and integrates better with Fabric’s Lakehouse architecture.
• Enable Incremental Refresh: This can drastically reduce refresh time if your data is time-partitioned.
• Use Data Pipelines: For better control over refresh sequencing and dependencies.
• Monitor Refresh Logs: Use refresh history and CSV logs to pinpoint slow phases (e.g., data read vs. model load).