Hidden Features

[binitafulpagare]

Hi everyone,

Sometimes the most valuable features of a platform are the ones that don't receive much attention.

In Microsoft Fabric, are there any features, capabilities, or workflows that you believe are underrated but have significantly improved your productivity?

It could be something related to notebooks, pipelines, shortcuts, monitoring, governance, Power BI integration, or any other part of Fabric.

I'd love to discover features that experienced users rely on but aren't discussed very often.

Thanks in advance for sharing your recommendations.

[nbleonhard]

One feature that I feel like doesn't get a ton of attention is setting up custom spark pools for your specific ML workload.

For example, if you are using single-node ML libraries (such as scikit-learn) your training process won't distribute across a Spark cluster. It will only execute on a single node. Because of this, I recommend setting up your spark pools based on your libraries:

• For single-node libraries (scikit-learn, etc.): Create a custom pool configured with one appropriately sized node, and make sure to disable autoscale and dynamic allocation. This prevents wasting resources on idle cluster nodes.

• For distributed training: If you want to take full advantage of a multi-node Spark cluster, ensure you are actively using distributed, multi-node machine learning libraries such as SynapseML. Also FYI, XGBoost now provides native, official support for distributed training on Apache Spark clusters.

[binitafulpagare]

Hi @nbleonhard,

Thank you for sharing this valuable practical insight. I agree that custom Spark pool configuration is an often overlooked aspect of optimizing machine learning workloads in Microsoft Fabric.

Your distinction between single-node libraries like scikit-learn and distributed frameworks such as SynapseML and Spark-based XGBoost is especially helpful. Configuring Spark pools based on the actual training framework can significantly improve resource utilization while avoiding unnecessary compute costs.

I also appreciate your recommendation to disable autoscale and dynamic allocation for single-node workloads, as it's a simple but effective optimization that many practitioners may not consider.

Thank you for sharing your experience—practical implementation tips like these are extremely valuable for anyone building efficient machine learning solutions on Microsoft Fabric.

[v-kathullac]

Hi @binitafulpagare ,

Thank you for reaching out to Microsoft Fabric Community Forum, below are the few points which can resolve your questions.

• OneLake Shortcuts reduce data duplication by providing access to data across different storage locations without copying it.
• Direct Lake mode enables near real-time reporting with faster query performance by accessing data directly from OneLake.
• Deployment Pipelines simplify application lifecycle management by promoting Fabric items across Development, Test, and Production environments.
• Git Integration enables version control, collaboration, and easier management of Fabric artifacts.
• Monitoring Hub provides centralized monitoring for pipelines, notebooks, semantic model refreshes, and other Fabric workloads.
• Dataflows Gen2 offer reusable, low-code ETL capabilities with improved performance and seamless integration with OneLake.
• Notebook Scheduling automates Spark notebooks for recurring data processing tasks without requiring external schedulers.
• Lakehouse and Warehouse integration allows the same data to be accessed using both Spark and SQL, improving flexibility for different workloads.
• Data Activator automates alerts and actions based on business events, reducing manual monitoring.
• Built-in Power BI integration enables quick creation of semantic models and reports directly from Fabric data, minimizing data movement and simplifying analytics.

Thanks & Regards,

Chaithanya.

[binitafulpagare]

Hi @v-kathullac,

Thank you for the detailed response and for highlighting these Microsoft Fabric capabilities.

I found the combination of OneLake Shortcuts, Direct Lake, Deployment Pipelines, and Git Integration particularly interesting, as they seem to address many common enterprise challenges around data duplication, performance, collaboration, and application lifecycle management.

I have one follow-up question based on production deployments. Among these features, which ones have organizations typically adopted first, and which have delivered the most immediate business value? For example, do most teams begin with OneLake and Lakehouse, or do they prioritize Git Integration, Deployment Pipelines, and Monitoring Hub as their Fabric environment matures?

I'd also be interested in hearing from other community members about which Fabric feature has had the biggest impact on their day-to-day workflows and why.

Thank you again for your guidance and for sharing these valuable recommendations!