Forum Discussion
shivaazure
1 year agoNew Member
Loading data from Lakehouse to Snowflake using Copy data in Data Pipeline
I have source Lakehouse table and target snowflake, using datalake copy data trying to preview the source lakehouse data, but getting this error: The data type is not supported in Delta format....
Anonymous
1 year agoNot applicable
Hi shivaazure ,
The error message indicates that void type is not supported. I have the following suggestions:
Before writing to the Delta table, you can convert the data type of the column in the notebook:
from pyspark.sql.functions import col
from pyspark.sql.types import StringType
#Convert the column to string type
df = df.withColumn("Extended_Status", col("Extended_Status").cast(StringType()))
Then use the following code to create the Delta table:
df.write.mode("overwrite").format("delta").saveAsTable("your_table_name")
Try again to see if the error still occurs.
Best Regards,
Yang
Community Support Team
If there is any post helps, then please consider Accept it as the solution to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!