Forum Discussion

AnibalBenedetto's avatar
AnibalBenedetto
Frequent Visitor
2 years ago

Error using Dataflow activity in Pipeline with dynamic content

Hello!, seems to be an issue with pipelines. When you add a Dataflow “activity” and set it with dynamic content (introducing his Dataflow object ID) you can Validate, Save and Run correctly, but if y...
data engineering
Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi AndyDDC , AnibalBenedetto ,

I tried to repro the scenario and got the same error when giving the "Dataflow Object ID" dynamically.

 

Instead of hard coding the value we can give the input as a parameter as shown in the screenshots below:

 

 

You can open the pipeline now even after closing it without any error.

Can you please try this AnibalBenedetto  and confirm, if this works?
Thanks.

AnibalBenedetto's avatar
AnibalBenedetto
Frequent Visitor
2 years ago

Hello!, I want a dynamic value, so Parameter is not useful in this case. In any case, I get the same error.

 

 

 

 

 

 

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Thanks for the update AnibalBenedetto .
    At this time, we are reaching out to the internal team to get some help on this.
    We will update you once we hear back from them.
    Thanks 

  • ajarora's avatar
    ajarora
    Microsoft Employee
    2 years ago

    One useful thing to gather here for a repro will be the json of the corrupted pipeline. And then using that for repro.

  • AnibalBenedetto's avatar
    AnibalBenedetto
    Frequent Visitor
    2 years ago

    Hello, I had created a new Dataflow and then I created a Pipeline that calls to that Dataflow using its dataflowId.

     

    Here you have the json code from the pipeline (before closing it):

    {
    "name": "toDelete",
    "objectId": "d7943d13-dfb9-4ac8-8423-0b00662751a7",
    "properties": {
        "activities": [
            {
                "name": "Dataflow1",
                "type": "RefreshDataflow",
                "dependsOn": [],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "typeProperties": {
                    "dataflowId": {
                        "value": "712a0555-226f-49eb-b298-75038b965749",
                        "type": "Expression"
                    },
                    "workspaceId": "740736a8-b130-4ae4-bbef-819062cf67c5",
                    "notifyOption": "NoNotification"
                }
            }
        ],
        "annotations": [],
        "lastModifiedByObjectId": "a1844513-17a8-4328-aae5-9d80baf0a8dc",
        "lastPublishTime": "2023-11-16T17:15:30Z"
    }
}

     

    Thanks!

  • v-cboorla-msft's avatar
    v-cboorla-msft
    Microsoft Employee
    2 years ago

    Hi AnibalBenedetto 

     

    Apologies for the delay in response.
    Follow up to see if you have a resolution yet on your issue.
    In case if you have any resolution please do share that same with the community as it can be helpful to others.

    This might require a deeper investigation from our engineering team about your workspace, the pipeline details and the logic behind it to properly understand what might be happening. 
    In case if you didn't get any resolution.

    Please go ahead and raise a support ticket to reach our support team: https://support.fabric.microsoft.com/support

     

    Please provide the ticket number here as we can keep an eye on it.

     

    Thank you.