Forum Discussion
Predefined Spark resource profiles
Good qusestion!
If you want to validate this approach, a practical solution is to:
Use Fabric's Activity Runs or Spark History to measure duration with each profile.
Keep volume constant, switch profile, and compare metrics like CPU Time, Shuffle Read/Write, and Cached Memory.
There are no public benchmarks from Microsoft for these specific scenarios, but i believe they have based this on early adopter feedback and internal testing from Fabric preview days (i assume)
WriteHeavy consistently reduces latency during large ingestions and merges.
ReadHeavyForSpark shows noticeable improvements in transformation heavy pipelines, especially those with large joins.
ReadHeavyForPBI makes PBI DL reports slick and more stable under the load.
The rationale behind tagging each layer is commonly based on below:
Bronze Layer (Raw Ingestion)
Recommended: Use WriteHeavy
Data is typically appended.
You are not reading it often; transformations happen downstream.
Prioritise write throughput and ingestion latency.
Silver Layer (Cleansed, Business Logic Applied)
Decision Point: Depends on your operations per table.
If table is append only and used in fact pipelines (transactional facts?):
Use: WriteHeavy
Optimise ETL throughput, especially if you are reading directly from Bronze and writing enriched data.
If table is dim like (ex: SCD & lookups) and used across many pipelines:
Use: ReadHeavyForSpark
These tables are typically read heavy across many processes (joins, lookups).
The frequency and cost of reads outweigh the write overhead.
True for SCD 2, where point in time analysis needs frequent reads with filters.
Gold Layer (Consumption/Visualization)
Use: ReadHeavyForPBI
Designed for consumption.
Read latency impacts user experience.
Optimised for directquery and DirectLake queries in PBI.
Please 'Kudos' and 'Accept as Solution' if this answered your query.
Hi Anonymous ,
Thanks for raising this in the Fabric Community Forum and thanks to Vinodh247 for the detailed input.
As mentioned by Vinodh247 , a commonly observed pattern is to use WriteHeavy for ingestion-heavy Bronze, ReadHeavyForSpark for dimension tables in Silver that are frequently read or joined, and ReadHeavyForPBI in the Gold layer where reporting performance is a focus.
For the Silver layer, selecting between WriteHeavy and ReadHeavyForSpark often depends on how the tables are accessed. Tables that are primarily written in bulk and used in fact processing tend to follow the WriteHeavy approach. In contrast, dimension tables, particularly those with SCD logic or used across multiple pipelines, may benefit from ReadHeavyForSpark due to more read-intensive operations.
If you're evaluating these options, comparing Spark history or Activity Runs across different profiles on the same workload can help identify which setting works best in your context.
You can find the official documentation on resource profiles here:
Configure Resource Profile Configurations in Microsoft Fabric - Microsoft Fabric | Microsoft Learn
Hope this helps. Please reach out for further assistance.
Please consider marking the helpful reply as Accepted Solution to assist others with similar queries.
Thank you.