View Dataflow parameters from pipeline when Workspace and Dataflow values are parameterized
I am following the documentation at https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-parameterized-dataflow to parameterize my dataflow and pass in the parameters from my variable library. I am able to pass the parameters to the dataflow from the pipeline to the dataflow but once I try to parameterize the workspace id and dataflow id in the pipeline, the dataflow parameters option disappears.
I've found a temporary workaround by saving it both ways and then manually merging the source file, but every time I open the pipeline to make edits, any saves overwrite my work around.
Can the UI be updated to show the dataflow parameters when variables are used for the dataflow?
Passing Parameterized values to the dataflow:
parameterizing the dataflow connection breaks/hides the ability to pass the parameters to the dataflow:
Manually edited pipeline-content.json:
4 Comments
- miguel
Community Admin
Status changed:NewtoPlannedWe're working on this. At the moment, if you parameterize the Dataflow ID or the WorkspaceID then the script created will only run against Dataflow Gen2 and not Dataflow Gen2 (with CI/CD) which uses a different API. This is documented in the article below:
Dataflow activity - Microsoft Fabric | Microsoft Learn
Once that limitation is lifted, things will work as you've mentioned and this is a quick preview on how that would look like:
- I might've found it, looks like Q12026 https://roadmap.fabric.microsoft.com/?productid=a821f83f-dbd6-ee11-9079-000d3a310f67&product=datafactory
- miguelStatus changed:PlannedtoCompleted
This was completed around a month ago, but now you're able to parameterize your Dataflow activity and actually pass parameters to the Dataflow Gen2 (CI/CD) as seen below:
You won't have the grid automatically discover the parameters given that the values entered for WorkspaceID and DataflowID are dynamic values, but you can enter any parameters and their values (dynamic or not) in the parameters section.
Recent ideas
Fabric REST API endpoint for dataflow transactions should GET *sortable* results for a time-range
Dataflow refresh APIs seem to be the buggiest; the list of transactions on a GET for Fabric-dataflow, are not sorted by the latest. OData query options such as $top, $filter, $orderby do not work wit...
