topic Re: how to refer to a folder in Lakehouse using Pyspark in Data Science

how to refer to a folder in Lakehouse using Pyspark

Jeanxyz — Fri, 13 Sep 2024 07:59:26 GMT

Hi, I have written a function which read documents from a local folder, which works well. Now I need to move the script and the local folder to OneLake and I have difficulty to refer to the folder in Fabric. I have tried relative path and absf path, both failed. Any ideas?

Re: how to refer to a folder in Lakehouse using Pyspark

Anonymous — Fri, 13 Sep 2024 08:29:56 GMT

HI Jeanxyz,

I'd like to suggest you use the file API path to instead, you can 'right click' on file stored in the folder and copy its API path.

/lakehouse/default/Files/source_files/

Regards,

Xiaoxin Sheng

Re: how to refer to a folder in Lakehouse using Pyspark

frithjof_v — Sat, 14 Sep 2024 12:25:05 GMT

These two articles also contain some related information:

https://fabric.guru/how-to-mount-a-lakehouse-and-identify-the-mounted-lakehouse-in-fabric-notebook

https://fabric.guru/how-to-attach-a-default-lakehouse-to-a-notebook-in-fabric

Also see:

https://learn.microsoft.com/en-us/fabric/data-engineering/notebook-utilities