Forum Discussion
Overwrite a table with different schema in Dataflow Gen2
I have a T1 table in Lakehouse L1 that I have ingested with Dataflow Gen2. I have another Lakehouse L2 with a shortcut to T1 (S2).
My T1 table in source now has one more column, so I redo the ingest with Dataflow Gen2 to overwrite the T1 table in L1, but I can't, because the schema has changed (one new column). The only way I see is to delete T1 in L1 and reload it. But if I do this, all references to T1 by its id, like the shortcut itself, are unlinked.
How can I update the schema of a table by doing overwrite with Dataflow Gen2 so that the shortcut and table ids are kept?
7 Replies
Hi,
You should be able to change the schema in the destination settings.
This what I get in f64 capacity. In f2 capacity you should be able to refresh your schema.
Hope this helps.
I'm using Premium F64 capacity.
I can see Dynamic Schema option when I create a NEW table, but not for Existing table:
For Existing table I can only see Refresh destination schema option, but is not working, because I click on it and the destination schema keeps the same. My source new schema has one more column, and it's not being added to destination schema:
You should be able to see the option. The screenshot I gave you was from existing table. From transform environment.
