Forum Discussion
Dataflow Gen2 CI/CD activity drops parameter values on save.
Summary
In a Data Pipeline, the Dataflow Gen2 (CI/CD) activity is not persisting its parameter values. Any value I enter in the activity's parameter section — whether dynamic content or a plain static literal — is silently discarded when the pipeline definition is saved. The binding disappears as soon as I close and reopen the pipeline, even without running it. Every other part of the activity persists normally.
All the usual prerequisites and misconfiguration causes have been ruled out (details below). I believe this is a product bug in the Gen2 CI/CD Dataflow activity's parameter serialization.
Environment
Microsoft Fabric — Data Pipeline with a Dataflow activity
Dataflow Gen2 (CI/CD) (confirmed, not standard Gen2)
Workspace is CI/CD / git-enabled
"Enable parameters to be discovered and overridden for execution" is ON, and the Dataflow was re-published after enabling it
Steps to reproduce
Open the Data Pipeline and select the Dataflow Gen2 (CI/CD) activity.
In the activity's Parameters section, the public parameters auto-appear (correctly discovered from the published Dataflow).
Enter a value for a parameter — tested with both dynamic content and a plain static literal.
Commit the cell (click out / apply) and Save the pipeline. Save succeeds.
Close the pipeline (navigate away / close the tab) and reopen it.
Expected behavior
The parameter value should persist in the saved pipeline definition and still be present when the pipeline is reopened.
Actual behavior
The parameter value is gone on reopen — the activity looks as if no value was ever entered. This happens before any execution; simply closing and reopening the pipeline is enough to lose it.
What I have already ruled out
Dataflow flavor — Confirmed this is Dataflow Gen2 (CI/CD), not standard Gen2.
Public parameters — "Enable parameters to be discovered and overridden for execution" is on, and the Dataflow was re-published after enabling it.
Parameter discovery — Parameters auto-appear in the activity; I am not hand-typing the rows, so it is not a name/type mismatch.
Pipeline save itself works — Renaming the activity persists correctly after close/reopen, so the pipeline save mechanism is committing.
Not expression-specific — A plain static literal value is dropped too, not just dynamic content.
Not a client issue — Reproduced in a second browser (and a hard refresh).
Stale activity schema — Deleted the Dataflow activity and added a fresh one, re-selecting the Dataflow so it reads a clean schema. The new activity reproduces the same drop-on-save.
Scope of the failure
The fault is narrowly the Dataflow activity's parameter block specifically failing to serialize into the saved pipeline definition. Everything else about the activity (and the pipeline save in general) persists correctly.
Questions
Is this a known issue with the Dataflow Gen2 (CI/CD) activity parameter serialization?
Is there a current fix, a workaround, or a tracked bug/known-issue reference?
As a workaround I am considering authoring the parameter binding directly in the pipeline JSON via git, and/or passing the value through a control/config table in a Lakehouse that the Dataflow reads at runtime. Are these the recommended fallbacks, or is there a supported approach?
Hi,
Thanks for your response. I've tried most of the suggestions, but unfortunately none of them worked.
However, I found one approach that does work:
- Pass the dynamic content into the dataflow parameters within the activity.
- Save the pipeline.
- Keep it open or, if you close, avoid opening the pipeline again, and execute it only through a schedule or a trigger.
I noticed that when you save the pipeline, the dynamic parameter values are preserved. The issue seems to occur when the pipeline is reopened in the workspace (or when it's executed directly from the pipeline page). At that point, the dynamic content appears to be overwritten.
As a workaround, every time you need to modify the pipeline (which requires opening it in the workspace), you'll need to reassign the dynamic content to the parameters before saving again.
In summary, keep the pipeline open while making your changes, then save it and execute it only through the scheduling or triggering mechanisms.
5 Replies
Hi gabrielfagundes ,
I have seen multiple recent reports of exactly this on Dataflow Gen2 (CI/CD) activities, parameters get discovered fine but vanish from the UI on reopen even though the JSON briefly holds the values. It does not appear to be a documented product limitation; it looks much more like a pipeline UI serialization bug introduced in the last few weeks. There is no officially confirmed Known Issue entry for it yet, so I would not call it "resolved as expected behavior."
A few things that have helped people in the meantime:- Open the Git/JSON representation of the pipeline and check if the parameter values are actually written under the Dataflow activity's parameters block. If they are present in JSON but missing in the UI, it confirms the UI rehydration bug and you can commit/deploy from Git as a temporary unblock.
- Delete and re-add the Dataflow Gen2 (CI/CD) activity from scratch (don't clone), set parameter values last, save once, and avoid switching tabs before saving.
- Double-check parameter types match exactly (pipeline String → Dataflow Text, not Any). Type mismatches have caused silent drops in similar scenarios.
- As a more reliable workaround, skip passing values via the activity and have the Dataflow read from a small control/config table (Lakehouse/Warehouse) at runtime; pipeline only writes the row before invoking the dataflow.
- If repro is clean, raise a Microsoft Support ticket with the pipeline JSON before and after reopen, that's the fastest path to getting it tracked.
Hope one of these gets you unblocked soon, do share back what worked in case others land on this thread later.
Useful links:Dataflow activity in Fabric Data Factory
Use public parameters in Dataflow Gen2 (Preview)
Parameterized Dataflow Gen2 end-to-end tutorial
Dataflow Gen2 with CI/CD and Git integration
Related community thread – parameters disappear on reopenDid this answer hit the mark? A small Kudos goes a long way — it fuels the community and encourages more folks to contribute.
Got what you needed? Marking this as the Accepted Solution helps others land on the right answer faster when they search for the same thing.
Hi,
Thanks for your response. I've tried most of the suggestions, but unfortunately none of them worked.
However, I found one approach that does work:
- Pass the dynamic content into the dataflow parameters within the activity.
- Save the pipeline.
- Keep it open or, if you close, avoid opening the pipeline again, and execute it only through a schedule or a trigger.
I noticed that when you save the pipeline, the dynamic parameter values are preserved. The issue seems to occur when the pipeline is reopened in the workspace (or when it's executed directly from the pipeline page). At that point, the dynamic content appears to be overwritten.
As a workaround, every time you need to modify the pipeline (which requires opening it in the workspace), you'll need to reassign the dynamic content to the parameters before saving again.
In summary, keep the pipeline open while making your changes, then save it and execute it only through the scheduling or triggering mechanisms.
Hey gabrielfagundes ,
Thanks for coming back and confirming what worked on your end — really appreciate you sharing the exact steps.
Your finding actually strengthens the theory that this is an authoring UI / rehydration issue rather than a runtime or parameter-discovery problem. The dynamic bindings clearly serialize correctly on save (schedule/trigger runs pick them up), but something in the reopen path is dropping or overwriting them before the next save.
For anyone else hitting this, the practical workaround is:- Set the parameter values on the Dataflow Gen2 activity.
- Save the pipeline.
- Do not reopen or edit the pipeline unless necessary.
- Trigger runs only via a schedule or pipeline trigger — not from the pipeline authoring page.
- If you do have to reopen it, reassign the parameter bindings before saving again.
One suggestion: if you can, capture the pipeline JSON right after save and again after a reopen (before re-saving). That before/after diff is a very clean, reproducible case to attach to a Microsoft Support ticket — it makes it much easier for the product team to isolate the layer where the values are being dropped.
If my earlier reply helped point you toward the right area, please consider marking it as the Accepted Solution so others running into the same issue can find the thread faster.
Hi gabrielfagundes
We wanted to follow up to check if you’ve had an opportunity to review the previous responses. If you require further assistance, please don’t hesitate to let us know.
