Looking for the best way to manage schema drift in Fabric ADF

[lwalton]

Does anyone know if this (ref link below) is the best process and also is it in the Fabric ADF resource?  I know Fabric ADF doesn't have all of the features native ADF has.  Thanks

https://learn.microsoft.com/en-us/azure/data-factory/concepts-data-flow-schema-drift

[Anonymous · Accepted solution]

Hi @lwalton ,

Schema Drift would be the best way to handle if we think data source keeps on changing and not constant.
There can be many situations possible, column added, column dropped, or column type changed.

Currently Schema Drift is possible through Azure Data Factory Mapping Dataflows, but they are not available in Fabric Data Factory.

Perhaps you want to try out Fabric Dataflows and see if it applies as it is?

In terms of what copy activity allows,

• If your destination table already exists, and the data you are writing has a column missing, it will be defaulted to null (default value) when writing to destination.
• If there is a new column, or if a column is not typecastable to the destination type, then this is treated as a bad row, and you can either skip writing this bad row (and log it into a temporary storage to be processed later), or fail the operation (the default).
  
  

Incase if that doesn't help, appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

Hope this helps. Please let me know if you have any further queries.

[Anonymous]

Hi @lwalton ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered. Otherwise, will respond back with the more details and we will try to help .

[Anonymous]

Hi @lwalton ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered. Otherwise, will respond back with the more details and we will try to help .