topic Re: Issue with parameterised Dataflow Gen2 activity in Data Engineering

Issue with parameterised Dataflow Gen2 activity

Anonymous — Wed, 18 Jun 2025 11:46:09 GMT

Problem Statement:

A published Dataflow Gen2 that contains parameters is not exposing those parameters to the Dataflow Gen2 activity within a Fabric Pipeline. The pipeline editor shows 'This dataflow has no parameters', even though they are defined and saved in the Dataflow.

Error Message When Pipeline is Run:

There was a problem refreshing the dataflow: 'Refresh with parameters is not supported for non-parametric dataflows'. Error code: ParametersNotSupported.

Troubleshooting Steps Already Performed:

The Dataflow (df_Update_KQL_Watermark) runs successfully when refreshed manually from the Dataflow editor itself. The manual run correctly writes data to the KQL database.
The Dataflow has been repeatedly saved and published after the parameters (pPipelineName, pNewWatermarkValue) were added.
We have waited for significant time (more than 5 minutes) after saving to account for any service latency.
The Dataflow Gen2 activity was deleted from the pipeline and a brand new one was created. The new activity still fails to detect the parameters when the Dataflow is selected.

Has anyone faced this issue?

Re: Issue with parameterised Dataflow Gen2 activity

wardy912 — Wed, 18 Jun 2025 13:13:29 GMT

I have faced this issue before and couldn't fix unfortunately, but as a workaround I used a notebook and I've never looked back.
With notebooks you can obtain a much more granular and deliberate set of data transformations.
You can also add base parameters to the notebook pipeline item which forces the pipeline to use it once referenced in your notebook, this is not available in dataflow gen2.
If notebooks are an option for you, I'd take that approach to avoid this issue.

To troubleshoot your dataflow gen2, even though your parameters are defined, make sure that the parameters are referenced in at least one of the queries, without this the pipeline may ignore it.

Re: Issue with parameterised Dataflow Gen2 activity

anon97242 — Wed, 18 Jun 2025 21:59:18 GMT

When you specify your DataflowGen2 with parameters in the Pipeline Dataflow Activity. you have to manually add the parameters. They do not automatically appear unfortunately.

Re: Issue with parameterised Dataflow Gen2 activity

v-aatheeque — Thu, 19 Jun 2025 09:32:35 GMT

Hi @Anonymous ,
@anon97242 @wardy912 Thanks for your quick response , In addition to that ,

Parameters in Dataflow Gen2 are part of a preview feature called public parameters. This mode only works with dataflows that have CI/CD support enabled. If your dataflow isn’t set up this way, the parameters won’t be exposed to the pipeline.

Reference : https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-parameters

Hope this helps!!

If this post was helpful, please consider marking Accept as solution to assist other members in finding it more easily.

If you continue to face issues, feel free to reach out to us for further assistance!