Forum Discussion

BettinaEK's avatar
BettinaEK
Advocate I
5 months ago
Solved

Dataflow Gen2 (CI/CD) Data Pipeline Activity Fails when Dynamic Parameters Passed from Data Pipeline

Hi Fellow Fabricators,
 
This one gave me a nasty headache, so I'm posting here to help others with the same issue.
 
Error code: 20302
Failure type: User configuration issue
Details
 Dataflow refresh job failed with status: Failed. Failure reason: { requestId: [guid], errorCode: 105004, message: . Learn more at https://go.microsoft.com/fwlink/?linkid=2314100 }
 
The error code Troubleshoot guide is generic and the failure reason message is missing.
 
High-level config details
 
DataflowGen2 (CI/CD):
  1. public parameters mode enabled
  2. runs successfully when (using "current value" in parameters)
    1. executed manually (save, run & close)
    2. refreshed (manually) from UI
  3. run fails 
    1. when called from a dataflow activity in a data pipeline, and
    2. pipeline passes 3 dynamic parameters into dataflow
 
MICROSOFT RESOURCES
"When selecting a Dataflow Gen2 with CI/CD that has the public parameters mode enabled, a Dataflow parameters section is displayed which lists all the available parameters, their types, and default values for the selected Dataflow."
 
 
SOLUTION
 
After a lot of config tweaking to identify the root cause, it was a simple data typing issue (of course!!).
 
Parameters in data pipeline items must be strongly typed - mine were of type [String]
The DFG2 parameters were of type Any - this is usually really forgiving.
 
By changing the data type of all parameters in DFG2 to [Text] the DPL activity ran perfectly.
 
NOTES: 
  1. The error could be much more meaningful - and indeed the documentation should be updated to make this data type match requirement clear. 
  2. Dataflows and Data Pipelines have different development stories:
    1. Dataflows have evolved considerably since their origin in Excel Power Query (aka Data Explorer)
    2. Fabric data pipelines began life in Azure Data Factory - with some Azure Synapse pipeline DNA.
  3. It is clear that these Fabric items are still not fully consistent - not least because types String and Text are not interchangeable/identical at a technical level 😛
  • SOLUTION: Parameter data types must align in DFG2 & DPL
     
    After a lot of config tweaking to identify the root cause, it was a simple data typing issue (of course!!).
     
    Parameters in data pipeline items must be strongly typed - mine were of type [String]
    The DFG2 parameters were of type Any - this is usually really forgiving.
     
    By changing the data type of all parameters in DFG2 to [Text] the DPL activity ran perfectly.
     
    NOTES: 
    1. The error could be much more meaningful - and indeed the documentation should be updated to make this data type match requirement clear. 
    2. Dataflows and Data Pipelines have different development stories:
      1. Dataflows have evolved considerably since their origin in Excel Power Query (aka Data Explorer)
      2. Fabric data pipelines began life in Azure Data Factory - with some Azure Synapse pipeline DNA.
    3. It is clear that these Fabric items are still not fully consistent - not least because types String and Text are not interchangeable/identical at a technical level : p

2 Replies

  • SOLUTION: Parameter data types must align in DFG2 & DPL
     
    After a lot of config tweaking to identify the root cause, it was a simple data typing issue (of course!!).
     
    Parameters in data pipeline items must be strongly typed - mine were of type [String]
    The DFG2 parameters were of type Any - this is usually really forgiving.
     
    By changing the data type of all parameters in DFG2 to [Text] the DPL activity ran perfectly.
     
    NOTES: 
    1. The error could be much more meaningful - and indeed the documentation should be updated to make this data type match requirement clear. 
    2. Dataflows and Data Pipelines have different development stories:
      1. Dataflows have evolved considerably since their origin in Excel Power Query (aka Data Explorer)
      2. Fabric data pipelines began life in Azure Data Factory - with some Azure Synapse pipeline DNA.
    3. It is clear that these Fabric items are still not fully consistent - not least because types String and Text are not interchangeable/identical at a technical level : p
  • Hi BettinaEK,

    We sincerely appreciate your efforts in sharing this valuable information with the community. It will benefit other community members who are seeking similar guidance.

    If you have any further queries, please feel free to contact the Microsoft Fabric community.

    Thank you.