Forum Discussion
Advocate IHow to handle schema drift?
Advocate IProbably all of the above. Current 'datafactory' has checkbox to handle. I see nothing like this in Fabric. This was the basis of my question.
Microsoft EmployeeWhat you are referring to is possible through Azure Data Factory Mapping Dataflows, but they are not available in Fabric. Perhaps you want to try out Fabric Dataflows and see if it applies as it is ?
In terms of what copy activity allows, if your destination table already exists, and the data you are writing has a column missing, it will be defaulted to null (default value) when writing to destination. If there is a new column, or if a column is not typecastable to the destination type, then this is treated as a bad row, and you can either skip writing this bad row (and log it into a temporary storage to be processed later), or fail the operation (the default).
Is the ADF Mapping Dataflow coming to Fabric ?
We have the same kind of requirement with json files as source, evolving with new attributes, we need to have the schema drift availableAny answer on this? We have the same requirements and need pipelines to be able to handle schema drift as it is under ADF.
