Hi @lwalton ,
Schema Drift would be the best way to handle if we think data source keeps on changing and not constant.
There can be many situations possible, column added, column dropped, or column type changed.
Currently Schema Drift is possible through Azure Data Factory Mapping Dataflows, but they are not available in Fabric Data Factory.
Perhaps you want to try out Fabric Dataflows and see if it applies as it is?
In terms of what copy activity allows,
- If your destination table already exists, and the data you are writing has a column missing, it will be defaulted to null (default value) when writing to destination.
- If there is a new column, or if a column is not typecastable to the destination type, then this is treated as a bad row, and you can either skip writing this bad row (and log it into a temporary storage to be processed later), or fail the operation (the default).
Incase if that doesn't help, appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.
Hope this helps. Please let me know if you have any further queries.
