Forum Discussion
Dataflow Gen2 CI/CD activity drops parameter values on save.
Hi,
Thanks for your response. I've tried most of the suggestions, but unfortunately none of them worked.
However, I found one approach that does work:
- Pass the dynamic content into the dataflow parameters within the activity.
- Save the pipeline.
- Keep it open or, if you close, avoid opening the pipeline again, and execute it only through a schedule or a trigger.
I noticed that when you save the pipeline, the dynamic parameter values are preserved. The issue seems to occur when the pipeline is reopened in the workspace (or when it's executed directly from the pipeline page). At that point, the dynamic content appears to be overwritten.
As a workaround, every time you need to modify the pipeline (which requires opening it in the workspace), you'll need to reassign the dynamic content to the parameters before saving again.
In summary, keep the pipeline open while making your changes, then save it and execute it only through the scheduling or triggering mechanisms.
Hi gabrielfagundes ,
I have seen multiple recent reports of exactly this on Dataflow Gen2 (CI/CD) activities, parameters get discovered fine but vanish from the UI on reopen even though the JSON briefly holds the values. It does not appear to be a documented product limitation; it looks much more like a pipeline UI serialization bug introduced in the last few weeks. There is no officially confirmed Known Issue entry for it yet, so I would not call it "resolved as expected behavior."
A few things that have helped people in the meantime:
- Open the Git/JSON representation of the pipeline and check if the parameter values are actually written under the Dataflow activity's parameters block. If they are present in JSON but missing in the UI, it confirms the UI rehydration bug and you can commit/deploy from Git as a temporary unblock.
- Delete and re-add the Dataflow Gen2 (CI/CD) activity from scratch (don't clone), set parameter values last, save once, and avoid switching tabs before saving.
- Double-check parameter types match exactly (pipeline String → Dataflow Text, not Any). Type mismatches have caused silent drops in similar scenarios.
- As a more reliable workaround, skip passing values via the activity and have the Dataflow read from a small control/config table (Lakehouse/Warehouse) at runtime; pipeline only writes the row before invoking the dataflow.
- If repro is clean, raise a Microsoft Support ticket with the pipeline JSON before and after reopen, that's the fastest path to getting it tracked.
Hope one of these gets you unblocked soon, do share back what worked in case others land on this thread later.
Useful links:
Dataflow activity in Fabric Data Factory
Use public parameters in Dataflow Gen2 (Preview)
Parameterized Dataflow Gen2 end-to-end tutorial
Dataflow Gen2 with CI/CD and Git integration
Related community thread – parameters disappear on reopen
Did this answer hit the mark? A small Kudos goes a long way — it fuels the community and encourages more folks to contribute.
Got what you needed? Marking this as the Accepted Solution helps others land on the right answer faster when they search for the same thing.
Hi,
Thanks for your response. I've tried most of the suggestions, but unfortunately none of them worked.
However, I found one approach that does work:
- Pass the dynamic content into the dataflow parameters within the activity.
- Save the pipeline.
- Keep it open or, if you close, avoid opening the pipeline again, and execute it only through a schedule or a trigger.
I noticed that when you save the pipeline, the dynamic parameter values are preserved. The issue seems to occur when the pipeline is reopened in the workspace (or when it's executed directly from the pipeline page). At that point, the dynamic content appears to be overwritten.
As a workaround, every time you need to modify the pipeline (which requires opening it in the workspace), you'll need to reassign the dynamic content to the parameters before saving again.
In summary, keep the pipeline open while making your changes, then save it and execute it only through the scheduling or triggering mechanisms.
Hey gabrielfagundes ,
Thanks for coming back and confirming what worked on your end — really appreciate you sharing the exact steps.
Your finding actually strengthens the theory that this is an authoring UI / rehydration issue rather than a runtime or parameter-discovery problem. The dynamic bindings clearly serialize correctly on save (schedule/trigger runs pick them up), but something in the reopen path is dropping or overwriting them before the next save.
For anyone else hitting this, the practical workaround is:- Set the parameter values on the Dataflow Gen2 activity.
- Save the pipeline.
- Do not reopen or edit the pipeline unless necessary.
- Trigger runs only via a schedule or pipeline trigger — not from the pipeline authoring page.
- If you do have to reopen it, reassign the parameter bindings before saving again.
One suggestion: if you can, capture the pipeline JSON right after save and again after a reopen (before re-saving). That before/after diff is a very clean, reproducible case to attach to a Microsoft Support ticket — it makes it much easier for the product team to isolate the layer where the values are being dropped.
If my earlier reply helped point you toward the right area, please consider marking it as the Accepted Solution so others running into the same issue can find the thread faster.Hi
Thanks for sharing your findings and the detailed workaround. We appreciate you taking the time to document the behavior you observed.It's helpful to know that the dynamic parameter values are retained after saving the pipeline, but can be overwritten when the pipeline is reopened. Your workaround of reassigning the dynamic content before saving and executing the pipeline through a schedule or trigger should be valuable for others encountering the same behavior while the issue is being investigated.
Thank you again for sharing your experience with the community.