Forum Discussion

suriaprakash's avatar
suriaprakash
Frequent Visitor
2 years ago
Solved

Notebook Run Is Failing Continuously

Hi,   The notebook in my pipeline is continuously failing despite multiple retries. Occasionally, it runs fine. This issue started on 07/18 at midnight PT and still persists as of 07/19 at midnight...
Dataflow
Need Help
  • suriaprakash's avatar
    suriaprakash
    2 years ago

    The problem got sorted out automatically. New spark pool with unlimited executors didn't solve the issue. Looks like a Fabric issue which happened coninously for a day or so.

SaiTejaTalasila's avatar
SaiTejaTalasila
Super User
2 years ago

Hi suriaprakash ,

 

Check your workspace settings and check whether the number of executers are auto or limited.If it is limited change it to auto and increase executers to max.Check spark UI incase if you are using spark programming it will show more details.If it is still not working download your report and create a new one and run it and check.And check for any changes in the source data type.

 

 

 

I hope it will be helpful.

 

Thanks,

Sai Teja 

  • suriaprakash's avatar
    suriaprakash
    Frequent Visitor
    2 years ago

    Thanks SaiTejaTalasila for your prompt response!

    There's no changes to either the pipeline or the datatypes recently. Also, I'm using the default Spark pool and so the executors are not limited. But I'll try to create a new Spark pool with no limit on executor count and try to run again. Will keep you posted.