Loading data from Lakehouse to Snowflake using Copy data in Data Pipeline

Hi shivaazure ,

 

The error message indicates that void type is not supported. I have the following suggestions:

 

Before writing to the Delta table, you can convert the data type of the column in the notebook:

from pyspark.sql.functions import col
from pyspark.sql.types import StringType

#Convert the column to string type
df = df.withColumn("Extended_Status", col("Extended_Status").cast(StringType()))

 

Then use the following code to create the Delta table:

df.write.mode("overwrite").format("delta").saveAsTable("your_table_name")

 

Try again to see if the error still occurs.

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Hi shivaazure ,

 

Is my follow-up just to ask if the problem has been solved?

 

If so, can you accept the correct answer as a solution or share your solution to help other members find it faster?

 

Thank you very much for your cooperation!

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!

Hi shivaazure ,

 

Is my follow-up just to ask if the problem has been solved?

 

If so, can you accept the correct answer as a solution or share your solution to help other members find it faster?

 

Thank you very much for your cooperation!

 

Best Regards,
Yang
Community Support Team

 

If there is any post helps, then please consider Accept it as the solution  to help the other members find it more quickly.
If I misunderstand your needs or you still have problems on it, please feel free to let us know. Thanks a lot!