Forum Discussion
Error writing table to Warehouse from notebook
- 5 months ago
Hi reubtfp,
The behavior you're seeing is expected with the current .synapsesql() connector. Even if you use mode("overwrite"), it only replaces the data in the target Warehouse table and doesn't support schema overwrite or evolution. The option("overwriteSchema","true") parameter isn't supported with .synapsesql(), which is why you get the “Schemas are not equal” error in both cases.
With this connector, the source DataFrame schema must match the target Warehouse table schema exactly—any differences in columns, data types, order, or nullability will cause the write to fail, regardless of the overwrite mode. This is a known connector limitation and isn't related to the runtime version.
To resolve the issue, make sure your DataFrame schema matches the target table before writing. If you expect schema changes, you'll need to drop and recreate the target table before writing. Alternatively, you can write to a Lakehouse (Delta table) first, where schema overwrite is supported, and then load the data into the Warehouse after aligning the schema.
Thank you.
Actually, this still doesn't work. I think it's a bug.
both of;
df.write.mode("overwrite").synapsesql(full_dest)