MS Fabric Notebook Spark Reading Data from Semantic Model

Hello mybarbie9917_LI

Thank you nilendraFabric  for your response!

Thank you for reaching out to the Microsoft Fabric Community. We understand you are experiencing a significant performance drop when querying a Power BI Semantic Model via Spark in Fabric.

This issue could be due to Fabric updates, Spark settings, or inefficient query execution. Since it worked in the past but has slowed down recently, it might be caused by Fabric capacity overload. If many users are running workloads, the capacity can be exhausted, slowing everything down.Could you please try the following methods:

• Microsoft Fabric upgraded to Runtime 1.3 with Apache Spark 3.5, bringing major library updates. If you use custom or overridden libraries, they may cause compatibility issues. Reinstalling libraries is recommended.
  
  Microsoft Fabric January 2025 update | Microsoft Fabric Blog | Microsoft Fabric
  Runtime 1.3 in Fabric - Microsoft Fabric | Microsoft Learn
  Compute management in Fabric environments - Microsoft Fabric | Microsoft Learn

• Optimize Spark settings by adjusting executor cores, memory, and autoscaling. For better performance, try disabling autoscaling and using a fixed number of executors.

• Use monitoring tools like Spark UI to diagnose performance bottlenecks. These tools provide detailed insights into the execution of Spark jobs, helping identify stages that are taking longer than expected.
  Apache Spark monitoring overview - Microsoft Fabric | Microsoft Learn

If the issue persists, it may be helpful to open a support ticket with Microsoft Fabric for further investigation.

How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn 

If my response has resolved your query, please mark it as the Accepted Solution to assist others. Additionally, a 'Kudos' would be appreciated if you found my response helpful.

Thank you!