How to set up a Pipeline that is using Dataflow Gen 2 as a trigger?

[NovaBI]

Hello,

I try to achieve the following solution:

1. I have a Dataflow Gen 2 (CI/CD) that is updated when a certain event happens (done with Power Automate, works fine)

2. I want to build a Pipeline that is using this Dataflow as a starting point (on success, on completion, doenst matter for now). When it has run through, I want another Dataflow to be started.

I have set up a pipeline, but it looks like the process won't start when the first dataflow has finished (also tried it manually).

Do I miss something?

[v-hashadapu]

Hi @NovaBI , Hope you're doing fine. Can you confirm if the problem is solved or still persists? Sharing your details will help others in the community.

[v-hashadapu]

Hi @NovaBI , Thank you for reaching out to the Microsoft Community Forum.

We find the answer shared by @tayloramy  is appropriate. Can you please confirm if the solution worked for you. It will help others with similar issues find the answer easily.

Thank you @tayloramy  for your valuable response.

[tayloramy · Accepted solution]

Hi @NovaBI, 

Why not make the pipeline drive your run? 

Switch it up so that your power automate runs the pipeline instead of the dataflow directly using the HTTP connector. 

Have the first activity of the pipeline be to run the dataflow. That ensures that any subsequent activities in the pipeline run after the dataflow is completed. 

[cengizhanarslan]

A Fabric Data pipeline cannot “listen” to a Dataflow Gen2 refresh that happened outside the pipeline and then auto-continue. In other words:

• If Dataflow Gen2 is refreshed by Power Automate (or manually), the pipeline won’t auto-trigger off that completion event.

• Pipelines currently trigger by manual run / schedule / (some) triggers, but not “on Dataflow Gen2 completion” as an event hook.

If DF1 must be triggered by Power Automate:

1. Trigger DF1 refresh in Power Automate

2. Poll/wait until refresh status = Completed (Power Automate often needs a loop checking refresh history/status)

3. Then:

   • Start the pipeline, or

   • Trigger Dataflow 2 refresh directly

Key point: you must explicitly “call the next step”; Fabric pipeline won’t automatically detect DF1 finishing.

First, grab the two IDs you need from Fabric:

• workspaceId = the GUID in the workspace URL

• itemId = the pipeline item’s GUID (you can usually see it in the item URL / “Copy link”, or retrieve it via Fabric item APIs if needed)

Then in Power Automate:

1. Create a flow (trigger can be manual, scheduled, or “when Dataflow refresh completes” if you already have that event).

2. Compose Ids

3. Add an action that can call a REST API with Entra ID auth (commonly HTTP with Microsoft Entra ID / a custom connector / or an app registration using client credentials).

[NovaBI]

This is what I'm building now, just need to wait for the Power Automate Prem License as HTTP Request is a premium feature 🙂

[Zanqueta]

Hi @NovaBI 

You are on the right track with using Dataflows Gen2 and Pipelines, but there is an important limitation: Fabric Pipelines do not currently support Dataflow completion as a native trigger. Pipelines can orchestrate Dataflows, but they cannot “listen” for a Dataflow to finish unless you explicitly control the sequence inside the pipeline.

✅ Important Notes

• Pipelines cannot currently “wait” for an external Dataflow unless you use an API or orchestrate everything inside the pipeline.
• If CI/CD is involved, ensure the pipeline and Dataflows are in the same workspace or properly referenced.

If this response was helpful in any way, I’d gladly accept a 👍much like the joy of seeing a DAX measure work first time without needing another FILTER.

Please mark it as the correct solution. It helps other community members find their way faster (and saves them from another endless loop 🌀.

[NovaBI]

I am thinking about an HTTPS call via Power Automate, maybe that can be a workaround for now.

I.e. set up "if an email arrives" -> Delay (2m or so) -> HTTP Request (POST) for workspace and pipeline id.

Not sure if that is reasonable and I dont like the idea of a workaround too much, but I think there is no better option for now...