Export entire delta table from lakehouse

[Anonymous]

Is there any way to get a .csv onto my computer from a delta table in a lakehouse? I haven't found any viable approach so far despite trying.

[FabianSchut · Accepted solution]

One way to achieve this is to write a small Python script in a notebook that reads the delta table from your lakehouse and exports it to the Files section of your lakehouse. It may be convinient to make a pandas dataframe after you've read the delta table with spark and use pandas 'to_csv' functionality.
With the OneLake file explorer: https://www.microsoft.com/en-us/download/details.aspx?id=105222, you could view and copy this csv to your local directory on your computer.

[Anonymous]

Brilliant, it works. FYI, if I use

df.write.csv("Files/my_csv.csv")

to write the file, it keeps the delta table file structure, i.e. it makes a folder named "my_csv.csv". However, the file(s) are in csv, so for my purposes it just works.

[frithjof_v]

If you use Pandas to_csv(), instead of PySpark write.csv(), you will get a single CSV file.

https://learn.microsoft.com/en-us/fabric/data-science/read-write-pandas

Pandas cannot work with such large data volumes as Spark. 

https://www.reddit.com/r/MicrosoftFabric/s/7EloDpd8wI

If your amount of data is not too large volume, then I guess I would use Pandas for writing to a single csv.

[lbendlin]

But seriously, converting a Delta file back to CSV is a travesty. Can your application not work with Parquet?

[Anonymous]

My goals are beyond your understanding. 

¯\_(ツ)_/¯

[lbendlin]

The Delta tables are already in Parquet format. it would be counterproductive to convert that to CSV.

"the entire table"  - careful with that. Parquet supports versioning so you risk exporting all the prior versions too.

[Anonymous]

There is only one version.