Forum Discussion

SørenBrandt's avatar
SørenBrandt
Frequent Visitor
1 year ago
Solved

400 error when accessing lakehouse files from notebook

Hi all! Pretty much every day, I experience spurious errors when accessing (csv) files in a notebooks default lakehouse. By spurious, I mean that they may happen in one session, but then later the s...
notebook
  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi SørenBrandt 

     

    You may try reading the CSV file with ABFS path instead and check whether this will be more reliable. This could avoid failures in a scenario that the default lakehouse might be changed.

     

    Best Regards,
    Jing
    Community Support Team

Anonymous's avatar
Anonymous
Not applicable
1 year ago

I feel like there might be some factors or issues that would cause the notebook session to fail to recognize the default lakehouse sometimes. Using the ABFS path is more stable because it directly specifies the absolute path of the lakehouse, rather than relying on the default lakehouse.

 

To improve the stability and reliability of reading the default lakehouse, you might try using the %%configure command at the beginning of the notebook to explicitly specify a default lakehouse. This will take effect in the current session. This magic command is supported when running the notebook directly or as a notebook activity in a pipeline.

Reference: Spark session configuration magic command

 

Best Regards,
Jing