Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago

VS Code notebook - error - When converting spark df to pandas

Hi,    I seem to be getting a similiar error to the issue raised here. I have a sample script where I am reading data from a table in the lakehouse - it worked fine initially so I am not sure what...
data engineering
v-cboorla-msft's avatar
v-cboorla-msft
Microsoft Employee
2 years ago

Hi Anonymous 


We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others.
Otherwise, will respond back with the more details and we will try to help.


Thanks.

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi,

 

I have attempted the suggestions in the post you shared but the issue is still ongoing. 

 

  • I have uninstalled and reinstalled vs code + synapse extension 
  • I have reinstalled pyspark + spark-lighter-lib
  • I have tried searching for logs but it looks like the log files havent saved for a few days. 

Is there anything else that may be causing the issue? 

 

I have included below the code + output 

 

 

 

  • v-cboorla-msft's avatar
    v-cboorla-msft
    Microsoft Employee
    2 years ago

    Hi Anonymous 

     

    Apologies for the delay in reponse from my end. Inorder to understand the exact issue, it requires a deeper investigation from our engineering team about your workspace and the logic behind it to properly understand what might be happening. 

    Please go ahead and raise a support ticket to reach our support team: support-ticket 

    Please provide the ticket number here as we can keep an eye on it.

     

    Thanks.

    • v-cboorla-msft's avatar
      v-cboorla-msft
      Microsoft Employee
      2 years ago

      Hi Anonymous 

       

      Following up on my previous inquiry, have you had the opportunity to raise a support ticket regarding this issue. If so, could you please provide the ticket number here as we can keep an eye on it.


      Thanks.