topic Re: After Initial Deployment, How Can We Transfer Actual Data from Dev to Prod in Microsoft Fabric in Data Engineering

After Initial Deployment, How Can We Transfer Actual Data from Dev to Prod in Microsoft Fabric

kasun_rumesh — Mon, 25 May 2026 10:16:07 GMT

I have implemented a Microsoft Fabric data pipeline with only two environments: Development (Dev) and Production (Prod).

In the Dev environment, data is loaded and maintained through incremental API ingestion, and historical data has been loaded initially using Excel files.

After the initial deployment, I understand that only metadata is promoted from Dev to Prod via the deployment pipeline. However, the actual data is not transferred between environments.

My question is: what is the recommended approach to handle the movement or synchronization of daily incremental data from Dev to Prod in this two-layer setup?

Rebuilding the pipeline or reloading data in Prod from scratch is not feasible due to time and effort constraints.

Is there a standard pattern in Fabric for handling this scenario (e.g., separate ingestion pipelines per environment, shared storage, or controlled data promotion strategies)?

Re: After Initial Deployment, How Can We Transfer Actual Data from Dev to Prod in Microsoft Fabric

tayloramy — Mon, 25 May 2026 13:29:01 GMT

Hi @kasun_rumesh,

Generally, you do not want to move data between DEV and PRD.

In most set ups, DEV is connected to dev/tst systems, and PRD is connected to production systems, this is to avoid bogging down the production enviornment with development, and to comply with company privacy policies and local legsligation where applicable.

If you must use the same data for development and prod, what I would recommend is you load all your data to a "data" workspace that sits outside of dev/prd, and then use shortcuts to reference that data in dev and in prd.

This does carry the risk that something going wrong in dev can impact prod destructively, so be careful and ensure you have strong controls in dev.

Re: After Initial Deployment, How Can We Transfer Actual Data from Dev to Prod in Microsoft Fabric

GilbertQ — Mon, 25 May 2026 22:12:48 GMT

Hi @kasun_rumesh

There have been some great suggestions above. Another option you could do is to use the materialized lake views in order to transfer the data from your dev to prod environment. Here is the link. https://learn.microsoft.com/en-us/fabric/data-engineering/materialized-lake-views/overview-materialized-lake-view

Re: After Initial Deployment, How Can We Transfer Actual Data from Dev to Prod in Microsoft Fabric

kasun_rumesh — Thu, 28 May 2026 10:17:07 GMT

Yes, I agree with what you said. However, I am looking for a solution to transfer data between environments. I have also noticed that Lakehouse schemas are not carried over across environments.

Re: After Initial Deployment, How Can We Transfer Actual Data from Dev to Prod in Microsoft Fabric

tayloramy — Thu, 28 May 2026 13:38:40 GMT

Hi @kasun_rumesh,

There is no automated way to do that in Fabric. You will need to build pipelines or notebooks that move the data across.

Re: After Initial Deployment, How Can We Transfer Actual Data from Dev to Prod in Microsoft Fabric

kasun_rumesh — Thu, 28 May 2026 13:42:34 GMT

Are there any known issues when using the Copy Job activity to transfer data from a Lakehouse?

Re: After Initial Deployment, How Can We Transfer Actual Data from Dev to Prod in Microsoft Fabric

tayloramy — Thu, 28 May 2026 13:44:44 GMT

Hi @kasun_rumesh,

Not to my knoweldge. THe only known issue I see is related to warehouses and private link.
https://support.fabric.microsoft.com/known-issues/?product=Data%2520Factory&active=true&fixed=true&sort=published&issueId=1796