topic Re: Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min in Data Warehouse

Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min

Deera — Fri, 17 Apr 2026 16:31:12 GMT

I have two pipelines.

The first pipeline contains two activities: Lookup and ForEach. Inside the ForEach activity, I’m invoking a second pipeline.

The second (invoked) pipeline contains 6 activities, which connect to source SQL Servers via a gateway and load data into a Data Warehouse in Fabric.

Here’s the issue:
From the parent (driver) pipeline view, it shows that Pipeline 2 takes about 3 minutes to complete. However, when I open Pipeline 2 and sum the duration of the individual activities, the total comes to only 1 minute.

Where is the remaining 2 minutes being spent?

Re: Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min

lbendlin — Fri, 17 Apr 2026 23:52:58 GMT

They are spent spinning up the Spark instance.

Re: Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min

Deera — Sat, 18 Apr 2026 03:54:09 GMT

But i haven't use any spark or notebok activity in it.

Re: Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min

pallavi_r — Sat, 18 Apr 2026 14:50:28 GMT

Hi Deera ,

The extra minutes, could be mostly coming from overhead outside the actual activity execution things like pipeline initialization, some queueing before it starts, and gateway-related delays when establishing connections to on-prem SQL.

Thanks,

Pallavi

Re: Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min

rizalard0684 — Sun, 19 Apr 2026 08:07:14 GMT

Deera in short the remaining 2 minutes are spent on getting things ready and cleaning up.
Let me elaborate what these are:
1. Pipeline startup overhead
When Pipeline 1 calls Pipeline 2, Fabric doesn’t instantly start the first activity. Before any activity runs, Fabric has to spin up the pipeline runtime, resolve parameters and build the execution graph. This time counts as pipeline duration, but does not belong to any activity, so you won’t see it inside Pipeline 2.

2. Gateway + data movement “warm‑up”

Because you’re using on‑prem SQL Servers and connecting through a gateway, Fabric needs to wake up the data movement engine, establish gateway connections, validate connectivity and allocate compute for copy.

All this happens before the first copy activity actually starts.
3. Pipeline invocation overhead inside ForEach

Every time Execute Pipeline runs there is an orchestration time, context switching between pipelines and tracking state between parent and child. Might not be long but still count a bit.
In summary, Your activities really do take only about 1 minute but the extra ~2 minutes are orchestration + gateway + runtime overhead.

Re: Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min

R1k91 — Mon, 20 Apr 2026 12:13:37 GMT

use the monitor , as others say you could see Queue or something else before the actual transfer

if you're importing from supported sources you could also see "time at first byte" (SQLServer should be) that basically tells you how much time the copy waited for the first byte and it should be equal to the queue.

Re: Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min

v-aatheeque — Thu, 23 Apr 2026 11:53:37 GMT

Hi Deera

Have you had a chance to look through the responses shared earlier? If anything is still unclear, we’ll be happy to provide additional support.

Re: Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min

v-aatheeque — Tue, 28 Apr 2026 09:16:44 GMT

Hi Deera

We wanted to follow up to check if you’ve had an opportunity to review the previous responses. If you require further assistance, please don’t hesitate to let us know.

Re: Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min

Deera — Tue, 28 Apr 2026 10:08:11 GMT

Thanks everyone for your insights. I have changed to invoke legacy activity instead of invoke preview activity which reduces my timing to 1.5 minutes. Based on your comments i understand that the extra time taken for gateway connectivity. Thanks

Re: Pipeline Duration Mismatch – Parent Pipeline Shows 3mins but Child Activities Total Only 1Min

v-aatheeque — Tue, 28 Apr 2026 12:22:02 GMT

Hi Deera

Glad to hear that the performance has improved after switching to the legacy activity.
Based on our findings, the remaining time is expected due to gateway connectivity overhead.
Please let me know if you face any further issues. If this solution helped, kindly acknowledge it, as it may benefit others in the community facing similar issues.