Forum Discussion
Export entire delta table from lakehouse
Is there any way to get a .csv onto my computer from a delta table in a lakehouse? I haven't found any viable approach so far despite trying.
One way to achieve this is to write a small Python script in a notebook that reads the delta table from your lakehouse and exports it to the Files section of your lakehouse. It may be convinient to make a pandas dataframe after you've read the delta table with spark and use pandas 'to_csv' functionality.
With the OneLake file explorer: https://www.microsoft.com/en-us/download/details.aspx?id=105222, you could view and copy this csv to your local directory on your computer.
7 Replies
One way to achieve this is to write a small Python script in a notebook that reads the delta table from your lakehouse and exports it to the Files section of your lakehouse. It may be convinient to make a pandas dataframe after you've read the delta table with spark and use pandas 'to_csv' functionality.
With the OneLake file explorer: https://www.microsoft.com/en-us/download/details.aspx?id=105222, you could view and copy this csv to your local directory on your computer.If you use Pandas to_csv(), instead of PySpark write.csv(), you will get a single CSV file.
https://learn.microsoft.com/en-us/fabric/data-science/read-write-pandas
Pandas cannot work with such large data volumes as Spark.
https://www.reddit.com/r/MicrosoftFabric/s/7EloDpd8wI
If your amount of data is not too large volume, then I guess I would use Pandas for writing to a single csv.
The Delta tables are already in Parquet format. it would be counterproductive to convert that to CSV.
"the entire table" - careful with that. Parquet supports versioning so you risk exporting all the prior versions too.
