Get certified in Microsoft Fabric—for free! For a limited time, the Microsoft Fabric Community team will be offering free DP-600 exam vouchers. Prepare now
Hi there,
I'm trying to configure my spark session in a Fabric notebook using the magic command detailed here: https://learn.microsoft.com/en-us/fabric/data-engineering/author-execute-notebook#spark-session-conf...
I only need to configure the default lakehouse, which runs successfully in the notebook interactive mode:
The issue is that when I run this notebook in a pipeline activity, I get an error:
Details:
Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - Exception, Error value - Error while creating the livy session, RunId: 1db415da-c5cf-482b-b787-40311575ef19' :
Any ideas on what I'm doing wrong here? Help would be much appreciated,
Dominik
Hello @dominikh ,
I also suggest you to add the retry and retry interval option set and see if that helps . I have seen this error but for me its not consistent .
thanks
Himanshu
Hi @HimanshuS-msft,
Thank you for the suggestion. It hasn't helped unfortunately. 😕 I configured the notebook activity with 3 retries and they all failed with the same error:
Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - Exception, Error value - Error while creating the livy session
Where can I find the run logs on the notebook? When I go to 'More options' > 'Recent runs' I don't see this pipeline run above.
Sincerely,
Dominik
Hi @dominikh
Apologies for the issue you have been facing. Are you still facing this issue?
If yes I would ask you to raise a support ticket here. https://support.fabric.microsoft.com/en-US/support/
Please provide the details here for us to track internally.
Thanks
Hello @v-nikhilan-msft, thanks for the recommendation. I am still facing this issue, so I will raise a support ticket.
Hi @govindarajan_d,
Thank you for trying to reproduce the issue as well. I got this issue yesterday for the first time, so it has persisted for a day at least. I'll give it another try tomorrow and see what happens. I'll also check for any running sessions and end them if needed.
@dominikh I tried to reproduce the issue and got the same error.
Seems to have happened in old Synapse as well. Might be a temp issue as mentioned in the below link:
Check out the October 2024 Fabric update to learn about new features.
Learn from experts, get hands-on experience, and win awesome prizes.
User | Count |
---|---|
12 | |
7 | |
5 | |
3 | |
2 |
User | Count |
---|---|
22 | |
16 | |
13 | |
7 | |
5 |