topic Re: change a parameter value defined in a dataflow in a data pipeline in Pipelines

change a parameter value defined in a dataflow in a data pipeline

ludovicLaurent — Tue, 09 Jul 2024 05:54:43 GMT

Hello

i have a question about a data pipeline that call a dafaflow gen 2 in microsoft fabric

My dataflow gen 2 has a date paremeter , i fixed a default value

i would when i execute the data pipeline i want to change the value defined in the parameter

I try with a variable and set variable : but no effects , i run with the default value

have a solution to do this ?

thanks

Re: change a parameter value defined in a dataflow in a data pipeline

NandanHegde — Tue, 09 Jul 2024 08:56:44 GMT

Based on my understanding, as of now the Dataflow Gen 2 cannot be parameterized at rune time via data pipelines. So it would be hardcoded version itself

Re: change a parameter value defined in a dataflow in a data pipeline

ludovicLaurent — Tue, 09 Jul 2024 13:43:19 GMT

Hello

what do you mean with "hardcoded version itself"

Do you mean to change the json generated by the pipeline ? I try some change but with no effect

thanks

Re: change a parameter value defined in a dataflow in a data pipeline

Anonymous — Wed, 10 Jul 2024 05:23:58 GMT

Hi ludovicLaurent ,

As NandanHegde said, when you want to make changes to a parameter that has been defined by default, the system won't allow it. Because it's written with an underlying structure that doesn't support such operations.

But I think you can use pipeline variables or expressions to set parameter values dynamically instead of using fixed values. If you want to use variables, make sure that you have a "Set Variable" activity before the "Execute Data Flow" activity. In the Parameters tab of the Execute Data Flow activity, set the parameter value to the variable you defined.

You can also refer to the following document for more information: Parameterizing mapping data flows - Azure Data Factory & Azure Synapse | Microsoft Learn

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Re: change a parameter value defined in a dataflow in a data pipeline

frithjof_v — Wed, 10 Jul 2024 07:44:49 GMT

Perhaps as a workaround you can store the date parameter in a Lakehouse (e.g. as a cell value in a table), and have the Dataflow Gen2 read the date value from the Lakehouse table.

Then, if you update the date value in the Lakehouse table, it should get updated in the Dataflow Gen2 also.

Here is a forum thread which tells that passing parameters directly from Data pipeline to Dataflow Gen2 is not possible. https://community.fabric.microsoft.com/t5/Data-Pipelines/Pass-data-pipeline-parameter-to-dataflow-gen2/m-p/3264346

You can also vote for this Idea to increase the priority of implementing this functionality:

https://ideas.fabric.microsoft.com/ideas/idea/?ideaid=2f761603-a400-ee11-a81c-6045bdb98602

Re: change a parameter value defined in a dataflow in a data pipeline

Anonymous — Thu, 25 Jul 2024 09:31:32 GMT

Hi ludovicLaurent ,

Have you solved your problem? If so, can you share your solution here and mark the correct answer as a standard answer to help other members find it faster? Thank you very much for your kind cooperation!

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.