Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Fabric Data Days Monthly is back. Join us on March 26th for two expert-led sessions on 1) Getting Started with Fabric IQ and 2) Mapping & Spacial Analytics in Fabric. Register now

Dataflow Gen2 on multi-environment scenarios

I'm copy-pasting from my forum post:
https://community.fabric.microsoft.com/t5/Data-Pipeline/Pipeline-source-control-issue-Dataflow-Gen2-...

I’m experiencing an issue with Dataflow Gen2 and source control.
Right now, even with CI/CD update Dataflow Gen2 doesn't support multi-environment scenarios.

 

In short, the pipeline commits typeProperties with the Dataflow’s logicalId instead of the dataflowId.
Also, workspaceId is set to a placeholder value of "00000000-0000-0000-0000-000000000000".


Steps to reproduce:

1. Create a Dataflow Gen2.

2. In a pipeline, add a Dataflow activity and select the Dataflow created in the previous step.

3. Commit changes to source control.

4. Deploy changes to another workspace.

5. Try to run the pipeline - you’ll get an UnknownException.

6. Manually update workspaceId and dataflowId in pipeline-content.json to the correct values. Now the pipeline works fine.

7. Commit the corrected values to source control.

8. After this commit, Fabric forces another commit, again replacing the dataflowId with logicalId and resetting the workspaceId to the placeholder value.

(Note: logicalId comes from the Dataflow’s .platform file.)

Status: New