Forum Discussion

dominikh's avatar
dominikh
Frequent Visitor
2 years ago

Error configuring spark session with magic command in pipeline activity

Hi there,   I'm trying to configure my spark session in a Fabric notebook using the magic command detailed here: https://learn.microsoft.com/en-us/fabric/data-engineering/author-execute-notebook#sp...
data engineering
HimanshuS-msft's avatar
HimanshuS-msft
Microsoft Employee
2 years ago

Hello dominikh , 
I also suggest you to add the retry and retry interval option set and see if that helps . I have seen this error but for me its not consistent .


thanks 
Himanshu 

dominikh's avatar
dominikh
Frequent Visitor
2 years ago

Hi HimanshuS-msft,

 

Thank you for the suggestion. It hasn't helped unfortunately. 😕 I configured the notebook activity with 3 retries and they all failed with the same error:

 

Notebook execution failed at Notebook service with http status code - '200', please check the Run logs on Notebook, additional details - 'Error name - Exception, Error value - Error while creating the livy session

 

 

Where can I find the run logs on the notebook? When I go to 'More options' > 'Recent runs' I don't see this pipeline run above. 

 

Sincerely,

Dominik

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi dominikh 
    Apologies for the issue you have been facing. Are you still facing this issue?
    If yes I would ask you to raise a support ticket here. https://support.fabric.microsoft.com/en-US/support/
    Please provide the details here for us to track internally.
    Thanks

    • dominikh's avatar
      dominikh
      Frequent Visitor
      2 years ago

      Hello Anonymous, thanks for the recommendation. I am still facing this issue, so I will raise a support ticket.