Author: Santhosh Kumar Ravindran - Principal Product Manager
Modern data engineering teams are under constant pressure to deliver faster pipelines, lower costs, and predictable performance—while dealing with increasingly diverse workloads. In practice, one-size-fits-all Spark configurations rarely work. Ingestion pipelines, transformation jobs, interactive analytics, and BI consumption all place very different demands on compute.
Figure: Introduction to Resource Profiles Experience.
Figure: Recommendations generated based on user inputs.
What are Resource Profiles?
Resource Profiles are preconfigured, workload-aware compute profiles that optimize Spark environment settings based on how your data is read, written, and consumed.
Instead of tuning dozens of Spark properties, you select a profile aligned to your workload pattern—Fabric applies the optimal configuration automatically.
Now, Fabric Data Engineering supports profiles such as:
- Write-heavy – Optimized for high-throughput ingestion and transformation workloads.
- Read-heavy for Spark – Optimized for frequent Spark reads and interactive queries.
- Read-heavy for Power BI – Optimized for BI and SQL consumption on Delta tables.
Each profile encapsulates best-practice settings refined from real-world workloads and internal benchmarking—so you get predictable performance out of the box.
Why Resource Profiles?
Across enterprise customers evaluating Microsoft Fabric, a common challenge emerges when teams struggle to achieve consistent price-performance when running diverse Spark workloads on a single set of configurations.
Write-heavy ingestion jobs need high throughput and parallelism. Read-optimized workloads demand low-latency access for analytics, SQL, and Power BI. Balancing these requirements manually often leads to trial-and-error tuning, operational overhead, and suboptimal results.
Resource Profiles address this head-on by letting you declare what you’re trying to do and letting Fabric handle how the compute should be optimized.
Built for Medallion Architectures and Task Specific
Resource Profiles map naturally to modern medallion architectures:
- Bronze layer ingestion jobs benefit from write-heavy profiles with optimized parallelism and file layout.
- Silver layer transformations can use balanced or read-heavy Spark profiles depending on access patterns.
- Gold layer consumption scenarios, in Power BI, SQL Warehouses, and interactive analytics—benefit from read-optimized profiles.
By aligning compute behavior to each stage of the data lifecycle, teams can achieve better SLAs, faster pipelines, and lower cost variability without complex tuning.
Performance by default and Control when you need it
Resource Profiles are designed to be safe, transparent, and flexible:
- New Fabric workspaces default to a write-optimized profile, ensuring strong ingestion performance from day one.
- Profiles can be configured at the environment level and overridden dynamically when needed.
- Existing workloads are not disrupted teams remain in full control of when and how profiles are applied.
This approach delivers performance by default while preserving the control enterprise teams expect.
Next steps
Resource Profiles are available now in Microsoft Fabric Data Engineering. Get started by selecting a profile in your Environment settings and experience the difference in your next pipeline run.
For detailed guidance, refer to the Microsoft Fabric Data Engineering documentation.
We'd love to hear how Resource Profiles are working for your team. Share your feedback in the comment section or join the conversation in the Microsoft Fabric Community.
Microsoft Employee
