Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago
Solved

Schema mismatch error In Warehouse

Goal: Develop an end-to-end automated pipeline to extract data from an external source, transfer it to Microsoft Fabric Lakehouse, perform data cleaning using a pre-designed PySpark notebook, save ...
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi Anonymous ,

    Thanks for using Fabric Community.

    At present schema drift is not available in Fabric.
    Copy activity allows, if your destination table already exists, and the data you are writing has a column missing, it will be defaulted to null (default value) when writing to destination. If there is a new column, or if a column is not typecastable to the destination type, then this is treated as a bad row, and you can either skip writing this bad row (and log it into a temporary storage to be processed later), or fail the operation (the default).