Fail to create session for executing notebook

[lyjavier]

Hi all,

I have faced an issue that "Failed to create session for executing notebook" with notebook activity within a data pipeline

I check the monitoring hub and see there is not any jobs running at the moment.

What's wrong with this?

Thank you in advance!

[Anonymous]

Hi @lyjavier ,

Is my follow-up just to ask if the problem has been solved?

If so, can you accept the correct answer as a solution or share your solution to help other members find it faster?

Thank you very much for your cooperation!

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

[Anonymous]

Hi @lyjavier ,

Thanks for the reply from jwinchell40 .

This error is usually caused by the Spark session failing to start or the previous session is still running, so a new session cannot be created.

I have the following suggestions:

• If the previous session is still running, please stop it.
• As jwinchell40 said, I wonder if this is a random problem for you or a persistent problem? Please try to delete and recreate the notebook, re-run it in the pipeline, and check if it works.
• I also learned from other posts with the same problem that the failure may be due to the use of an outdated Spark Run time version, please check if the Notebook Run time version you are currently running is the latest version.

If you have any other questions, please feel free to contact me.

Best Regards,
Yang
Community Support Team

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

[lyjavier]

It's just a pipeline including 2 notebook activities with 2 parameter that will be sent to the first notebook as toggle parameters. Idk what is wrong with Fabric? It's so annoying tbh

[jwinchell40]

@lyjavier  - I sometimes get random failures like this and they normally just run fine the next time out when they are retried.  I have not been able to find a solution or root cause for these random failures yet.

Can you tell me more about the Pipeline and Notebook activity?  Any custom libraries being loaded?  Any parameters?  

Early on when I was adding Notebook steps to data pipelines; one thing I ran into is that some libraries (like Semanntic Link) wouldn't load when the Notebook was run from a Pipeline.  I was using the pip command do that.  There is a setting change you can make in the Fabric Pipeline (I will find it from one of our pipelines and post it here) but that fixed that issue.