Forum Discussion

DarSz's avatar
DarSz
Helper I
1 year ago

Pipeline source control issue: Dataflow Gen2 pipelines overwrite dataflowId with logicalId

Hi, I’m experiencing an issue with Dataflow Gen2 and source control. In short, the pipeline commits typeProperties with the Dataflow’s logicalId instead of the dataflowId. Also, workspaceId is set ...
Need Help
Pipeline
Vinodh247's avatar
Vinodh247
Super User
1 year ago

Did you try the RESTAPI? The Dataflow Gen2 pipeline activity doesn't support parameterized or dynamic inputs for workspaceId and dataflowId. It expects selections via dropdown only. If you manually edit those IDs (even with correct ones), Fabric will override them with the logicalId and zeroed workspaceId on the next commit, garbage in, garbage out. That’s by design, not a glitch.

 

Re: Unable to run Dataflow Gen 2 (CI/CD) by giving... - Microsoft Fabric Community

 

No timeline or ETA on a fix, this hasn’t acknowledged as urgent. So far, no roadmap or release notes hint that parameterized Dataflow activities for Gen2 will work any differently soon.

 

Please 'Kudos' and 'Accept as Solution' if this answered your query.

DarSz's avatar
DarSz
Helper I
1 year ago

I worked around the issue by using Notebooks instead of Dataflow Gen2.

It’s disappointing to hear this. It basically means Dataflows are unusable in multi-environment scenarios. In their current state, I can’t rely on them.