topic Slow notebooks in Pipelines

Slow notebooks

Anonymous — Mon, 14 Oct 2024 13:29:17 GMT

I've noticed that my notebook in Microsoft Fabric has suddenly become quite slow today. Interestingly, I encountered a similar issue before, and the next day everything returned to normal without any intervention.

I wanted to check if anyone else is experiencing the same performance issues or if there might be something specific affecting my environment.

Thanks,

Re: Slow notebooks

Anonymous — Tue, 15 Oct 2024 08:13:28 GMT

Hi Anonymous ,

The current server status is all normal, is there still slow?

Microsoft Fabric Support and Status | Microsoft Fabric

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Re: Slow notebooks

nomadica — Sun, 24 Nov 2024 08:07:57 GMT

Over the last few months of Fabric experience, and in general, notebooks were slow for me, moving from F16 to F64 didn't impact speed.

PySpark job relalted data quality analysis that reads from a config file, checks every row, and inserts rows in pass/failed table take over 30 minutes for a 700 rows table with 35 column.

Re: Slow notebooks

jagdishwagh0909 — Wed, 29 Jan 2025 13:06:04 GMT

I am facing the same issue when we are running notebook from fabrics It will take to much time for small operation as well. How we can improve notebook slowness in future.

Re: Slow notebooks

nomadica — Wed, 29 Jan 2025 14:03:59 GMT

in our case it was a combination of factor:

1- Cold run

2- Small semnatic model

3- Owner user of lakehouse was disabled (caused many more bad symptoms including SQL endpoint Access Denied error).

4- Too many tables in the lakehouse

5- Using custom pool instead of standard pool, which takes 3 minutes to Start

Apart from that, the notebook logic itself may require review (including if the notebook accesses multiple sources including documents in Lakeouse.

Bottom line, this requires multiple checks to answer 🙂