Git integration for Datawarehouse

Hi caseybks ,

Thank you for reaching out to the Microsoft fabric community forum and for sharing the details of your scenario. Also thank you Arul  for your helplful response.

I understand that you are using Git integration with Fabric CI/CD for Data Warehouse deployments and experiencing data loss in tables when schema changes, such as adding or removing columns, are merged from a feature branch into the develop branch and then synced to the development workspace. This occurs because, due to current limitations in Fabric Git integration for Data Warehouses, schema changes made through Git sync will recreate the underlying object to match the repository definition, which results in the data being cleared.

This limitation, which also applies to SQL databases in Fabric, is documented here: Source control in Fabric Data Warehouse – Limitations.

To address this, you can back up your data before making schema changes by using COPY INTO or exporting to staging or external storage, then reload the data after deployment. Keep schema and data operations separate by applying schema updates via Git and restoring data with post-deployment scripts in Fabric pipelines or Fabric-CICD. It's important to test these steps in a staging environment before moving to development or production. Since Git sync can't currently preserve table data during schema changes, these methods can help manage the impact until the platform provides a solution.

Hope this helps. Please reach out for further assistance.

Thank you.