Forum Discussion
Solved
notebook pyspark constant errors writing more than 2 files
I am trying to output many lakehouse tables to parquet files under the Files section within the lakehouse, but in pyspark notebook it can only seem to output 2 tables without failing, anything more t...
The data exists in the delta table in the lakehouse and it seems to display the data without issues - so I'm assuming the structure of the delta files are fine.
But it seems like a issue with data quality specifically column names with suprious characters like $ . " etc. But if it is already in the lakehouse as delta table don't understand why it can't output the data without failing. Anyway I had to clean up all the column names then it worked
Hi MangoMagic
Thank you for reaching out microsoft fabric community forum.
May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.
Thank you.