Forum Discussion
BettinaEK
5 months agoAdvocate I
Dataflow Gen2 (CI/CD) Data Pipeline Activity Fails when Dynamic Parameters Passed from Data Pipeline
Hi Fellow Fabricators,
This one gave me a nasty headache, so I'm posting here to help others with the same issue.
Error code: 20302
The error code Troubleshoot guide is generic and the failure reason message is missing.
Failure type: User configuration issue
Details
Dataflow refresh job failed with status: Failed. Failure reason: { requestId: [guid], errorCode: 105004, message: . Learn more at https://go.microsoft.com/fwlink/?linkid=2314100 }
High-level config details
DataflowGen2 (CI/CD):
- public parameters mode enabled
- runs successfully when (using "current value" in parameters)
- executed manually (save, run & close)
- refreshed (manually) from UI
- run fails
- when called from a dataflow activity in a data pipeline, and
- pipeline passes 3 dynamic parameters into dataflow
MICROSOFT RESOURCES
Passing parameters into a Dataflow Gen2 CI/CD from a Data Pipeline activity is supported: and referenced thus
"When selecting a Dataflow Gen2 with CI/CD that has the public parameters mode enabled, a Dataflow parameters section is displayed which lists all the available parameters, their types, and default values for the selected Dataflow."
SOLUTION
After a lot of config tweaking to identify the root cause, it was a simple data typing issue (of course!!).
Parameters in data pipeline items must be strongly typed - mine were of type [String]
The DFG2 parameters were of type Any - this is usually really forgiving.
By changing the data type of all parameters in DFG2 to [Text] the DPL activity ran perfectly.
NOTES:
- The error could be much more meaningful - and indeed the documentation should be updated to make this data type match requirement clear.
- Dataflows and Data Pipelines have different development stories:
- Dataflows have evolved considerably since their origin in Excel Power Query (aka Data Explorer)
- Fabric data pipelines began life in Azure Data Factory - with some Azure Synapse pipeline DNA.
- It is clear that these Fabric items are still not fully consistent - not least because types String and Text are not interchangeable/identical at a technical level 😛
- SOLUTION: Parameter data types must align in DFG2 & DPLAfter a lot of config tweaking to identify the root cause, it was a simple data typing issue (of course!!).Parameters in data pipeline items must be strongly typed - mine were of type [String]The DFG2 parameters were of type Any - this is usually really forgiving.By changing the data type of all parameters in DFG2 to [Text] the DPL activity ran perfectly.NOTES:
- The error could be much more meaningful - and indeed the documentation should be updated to make this data type match requirement clear.
- Dataflows and Data Pipelines have different development stories:
- Dataflows have evolved considerably since their origin in Excel Power Query (aka Data Explorer)
- Fabric data pipelines began life in Azure Data Factory - with some Azure Synapse pipeline DNA.
- It is clear that these Fabric items are still not fully consistent - not least because types String and Text are not interchangeable/identical at a technical level : p
2 Replies
- BettinaEKAdvocate ISOLUTION: Parameter data types must align in DFG2 & DPLAfter a lot of config tweaking to identify the root cause, it was a simple data typing issue (of course!!).Parameters in data pipeline items must be strongly typed - mine were of type [String]The DFG2 parameters were of type Any - this is usually really forgiving.By changing the data type of all parameters in DFG2 to [Text] the DPL activity ran perfectly.NOTES:
- The error could be much more meaningful - and indeed the documentation should be updated to make this data type match requirement clear.
- Dataflows and Data Pipelines have different development stories:
- Dataflows have evolved considerably since their origin in Excel Power Query (aka Data Explorer)
- Fabric data pipelines began life in Azure Data Factory - with some Azure Synapse pipeline DNA.
- It is clear that these Fabric items are still not fully consistent - not least because types String and Text are not interchangeable/identical at a technical level : p
- v-pnaroju-msftCommunity Support
Hi BettinaEK,
We sincerely appreciate your efforts in sharing this valuable information with the community. It will benefit other community members who are seeking similar guidance.
If you have any further queries, please feel free to contact the Microsoft Fabric community.
Thank you.