Forum Discussion

kangwen1's avatar
kangwen1
New Member
2 years ago
Solved

notebook queued time

When I tried to trigger the notebook by the pipeline, I always find that though the notebook execution time is 10 minutes, it's possible the queued duration is 2 minutes. Why are there this kind of ...
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi kangwen1 ,

     

    Thanks for the reply from frithjof_v .

     

    There are several possible reasons for queuing.

     

    If the resources required to execute the notebook are occupied, the job will be queued until the resources are free.

     

    There may be a limit to the number of concurrent runs that can be made, resulting in new jobs waiting in the queue.

     

    You can refer to the following documentation for more details on concurrency limits:

    Concurrency limits and queueing in Apache Spark for Fabric - Microsoft Fabric | Microsoft Learn

     

    To reduce queuing time, I have a few suggestions here.

     

    Running tasks during periods of low demand can help reduce queuing time, for example, most of the evening is more free than the morning.

     

    You can configure high concurrency mode for notebooks.

     

    This document provides steps on how to configure high concurrency mode for notebooks:

    Configure high concurrency mode for notebooks - Microsoft Fabric | Microsoft Learn

     

    If you have any other questions please feel free to contact me.

     

    Best Regards,
    Yang
    Community Support Team

     

    If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
    If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!