topic Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline in Pipelines

Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

Darshan22 — Tue, 06 May 2025 10:30:51 GMT

Hi Everyone,

I'm working with Microsoft Fabric pipelines and using a Variable Library to dynamically assign values to the Dataflow activity, specifically for workspaceId and dataflowId.

My Setup:

I've created a Variable Library with the following variables:
- env: contains the Workspace ID (GUID)
- movex_customers: contains the Dataflow ID (GUID)
- dataflow_type: contains the value "DataflowFabric"

In the pipeline Dataflow activity:

Workspace ID:
@pipeline().libraryVariables.env
Dataflow ID:
@pipeline().libraryVariables.movex_customers

The Problem:

When I use these variables, the Dataflow activity fails with the following error:

{
  "error": {
    "code": "InvalidRequest",
    "message": "Unexpected dataflow error"
  }
}

Fabric doesn't know it should treat the activity as a Dataflow, so it throws a generic internal error instead of a clear validation failure. This error goes away only when I add the following field to the activity configuration manually in JSON view:

"dataflowType": "DataflowFabric"

However, the Fabric pipeline UI does not expose a field for dataflowType when using dynamic inputs — only when selecting a Dataflow statically.

What I’ve Tried:

Variable substitution via expression:
- Created a variable: dataflow_type = "DataflowFabric"
- Referenced in activity: @pipeline().libraryVariables.dataflow_type
- → Field still missing in UI or has no effect
Set Variable / Script activity workaround:
- Used a Set Variable or Script activity to define:
```
resolved_dataflow_type = "DataflowFabric"
```
- Referenced: @variables('resolved_dataflow_type')
- → Still not picked up; there's no place to plug it in via UI
Hardcoding works:
- Manually typing Workspace ID and Dataflow ID → activity runs successfully
- Adding dataflowType via JSON works
- When dynamic expressions are used for IDs and dataflowType is not set, it fails with “Unexpected dataflow error”

What I'm Looking For:

Why is Fabric not recognizing the activity as a DataFlow?
How can I dynamically set dataflowType = "DataflowFabric" when using a Variable Library?
Is there a hidden property, advanced field, or workaround to inject this value in the UI?
Is there a future update planned to make dataflowType configurable from the designer?

Happy to share screenshots or test results if needed.

Thanks in advance!

Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

v-prasare — Wed, 07 May 2025 07:24:36 GMT

Hi Darshan22,

Please refer to below similar solved thread from community and let me know if this helps?

https://community.fabric.microsoft.com/t5/Data-Pipeline/dynamic-library-variables/m-p/4663725/highlight/true

Thanks,

Prashanth Are

MS Fabric community support

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query

Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

v-prasare — Mon, 12 May 2025 07:00:48 GMT

Hi Darshan22,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for your issue worked? or let us know if you need any further assistance here?

Thanks,

Prashanth Are

MS Fabric community support

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query

Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

v-prasare — Mon, 12 May 2025 10:22:47 GMT

Hi Darshan22,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for your issue worked? or let us know if you need any further assistance here?

Thanks,

Prashanth Are

MS Fabric community support

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly and give Kudos if helped you resolve your query

Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

Darshan22 — Mon, 12 May 2025 10:57:36 GMT

Hi v-prasare,

The solution provided focus on dynamically setting Library variables. Whereas in this case the Library variables are static, they do not change. This query concerns with dynamically defining the Dataflow id and Workspace Id using Library variables. So no, it doesn't really answer my question.

Thanks,
Darshan

Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

bsmithUI — Wed, 21 May 2025 20:05:49 GMT

I am running into this exact same problem, feels like it should work given you can dynamically populate the Dataflow ID...but something isn't wired correctly. I noticed the only real difference is the missing dataflowType = "DataflowFabric" when you compare the inputs.

Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

v-prasare — Thu, 22 May 2025 12:14:32 GMT

Hi Darshan22 ,If your blocked here in this scenario i suggest you to raise a support ticket here. so, that they can assit you in addressing the issue you are facing. please follow below link on how to raise a support ticket:

How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn

Thanks,

Prashanth

Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

CSERRA — Thu, 28 Aug 2025 10:18:34 GMT

Hi Darshan,

We are facing the same issue, did you find a solution? The thread mentions "solved" but it doesn't seem like it.

Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

MSProfessional — Thu, 04 Sep 2025 18:09:32 GMT

I am facing the same challenge. It would be helpful if the above error is added to the known issue(https://support.fabric.microsoft.com/known-issues/) so we can track the same properly and plan the deployment accordingly.

Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

MSProfessional — Thu, 04 Sep 2025 18:11:46 GMT

I am facing the same issue. It would be helpful if the known issue(https://support.fabric.microsoft.com/known-issues/) is updated with the same so we can track the same properly and plan the deployment accrodingly.

Re: Dataflow Not Running with dynamic variables from Variable Library in Fabric Pipeline

MSProfessional — Thu, 04 Sep 2025 18:26:19 GMT

The solution to the above issue can be manually adding below from "View" Menu > "Edit JSON code" option until it is fixed from MS.
"dataflowType": "DataflowFabric"