Forum Discussion
Deployment pipelines - Dataflows in deployed workspaces output to dev
Hi dolphinantonym,
When deploying Dataflows Gen2 using deployment pipelines, the dataflows keep their original connection and destination settings. As a result, even after moving from Dev to Test or Prod, they continue to write to the Lakehouse in the Dev workspace. Currently, Fabric does not provide an automated way to update all dataflow outputs in bulk during deployment.
To handle this, you can use Variable Libraries to maintain environment-specific configurations. Each stage in the pipeline (Dev, Test, Prod) can have its own active value set in the same variable library. These values let you reference environment-specific parameters—like workspace IDs, Lakehouse names, or table identifiers—without manual changes after each deployment. During runtime, the dataflow uses the active value set for the environment, ensuring the correct configuration is applied.
If you use Git-based source control, Variable Libraries are Git-enabled and versioned with your dataflows. When you create a workspace from a feature branch, you can select the active value set for that branch or environment, allowing flexible testing and management of configurations.
In the dataflow’s mashup.pq file, you can reference variables using Power Query functions like Variable.Value() or Variable.ValueOrDefault(). For example, Variable.Value("$(/**/My Library/Workspace ID)") gets the workspace ID dynamically. However, Variable Libraries can only control parameters used inside the query logic and cannot rebind the output sink set in the dataflow UI.
For larger environments with many dataflows, you can use Fabric REST APIs to automate rebinding. A script can loop through dataflows in the target workspace and update their destination properties, providing scalable automation to ensure deployed dataflows write to the correct Lakehouses after deployment.
Lifecycle Management of the Microsoft Fabric Variable library - Microsoft Fabric | Microsoft Learn
Use Fabric variable libraries in Dataflow Gen2 (Preview) - Microsoft Fabric | Microsoft Learn
Thank you.
By defining variables such as the Lakehouse or table name with environment-specific values, you can ensure dataflows write to the correct Lakehouse after deployment.
By "environment-specific", do you mean using the deployment pipeline to set the variables correctly for test and prod environments as part of deployment?
If so, is there a way to do the same thing for feature branches that are created from source control rather than via deployment pipelines?
Also, do you know where to insert the variable to the dataflow? I can't find where it is editable, and the only place I can see the output Workspace being referenced is in a file called "mashup.pq" when I look at my Dataflow in GitHub, but I can't see where to find/edit it from within Fabric.
Hi dolphinantonym,
When deploying Dataflows Gen2 using deployment pipelines, the dataflows keep their original connection and destination settings. As a result, even after moving from Dev to Test or Prod, they continue to write to the Lakehouse in the Dev workspace. Currently, Fabric does not provide an automated way to update all dataflow outputs in bulk during deployment.
To handle this, you can use Variable Libraries to maintain environment-specific configurations. Each stage in the pipeline (Dev, Test, Prod) can have its own active value set in the same variable library. These values let you reference environment-specific parameters—like workspace IDs, Lakehouse names, or table identifiers—without manual changes after each deployment. During runtime, the dataflow uses the active value set for the environment, ensuring the correct configuration is applied.
If you use Git-based source control, Variable Libraries are Git-enabled and versioned with your dataflows. When you create a workspace from a feature branch, you can select the active value set for that branch or environment, allowing flexible testing and management of configurations.
In the dataflow’s mashup.pq file, you can reference variables using Power Query functions like Variable.Value() or Variable.ValueOrDefault(). For example, Variable.Value("$(/**/My Library/Workspace ID)") gets the workspace ID dynamically. However, Variable Libraries can only control parameters used inside the query logic and cannot rebind the output sink set in the dataflow UI.
For larger environments with many dataflows, you can use Fabric REST APIs to automate rebinding. A script can loop through dataflows in the target workspace and update their destination properties, providing scalable automation to ensure deployed dataflows write to the correct Lakehouses after deployment.
Lifecycle Management of the Microsoft Fabric Variable library - Microsoft Fabric | Microsoft Learn
Use Fabric variable libraries in Dataflow Gen2 (Preview) - Microsoft Fabric | Microsoft Learn
Thank you.
Hello,
I would like to know if is there any new update for dataflows deploy process for 2026 ?
Best regard,
Eugenio Miliani
Hi jmiliani74,
As of 2026, there are no updates to the Dataflow Gen2 deployment process in Microsoft Fabric. Deployment pipelines still copy Dataflows Gen2 without changes, so the original connection and output destination remain the same when moving from Dev to Test or Prod.
Dataflow Gen2 does not support deployment rules yet, so there is no automatic way to rebind the output Lakehouse during deployment. This means dataflows will continue to write to the Dev workspace Lakehouse unless the destination is manually updated.
The current options are unchanged: for small environments, update the destination Lakehouse manually after deployment; for larger environments, use Fabric REST APIs to automate updating dataflow destinations in bulk. Variable Libraries can help manage environment-specific values in query logic, but cannot change the output sink set in the Dataflow UI.
There is no official announcement about upcoming changes to this behavior.
Thank you.
