topic Dependency and Parallel Processing in Pipelines

Dependency and Parallel Processing

Anonymous — Thu, 06 Mar 2025 11:26:31 GMT

i have four pipelines running in 2 streams.
each stream has 2 pipelines running in sequence.
if a pipeline fails then the next pipeline should not get executed.
However, this should not affect the one's running in parallel and they should continue.
Final status of the pipeline should be Failed.

Re: Dependency and Parallel Processing

NandanHegde — Thu, 06 Mar 2025 11:35:22 GMT

please provide a pictorial represent of your ask as that would be much more clearer rather than the above statements.
meaning when you say if a pipeline fails, is it the pipeline within the stream or the external pipeline etc?

Re: Dependency and Parallel Processing

Anonymous — Thu, 06 Mar 2025 11:36:53 GMT

Master Pipeline:
├── Pipeline A (Runs in parallel)
│ └── Pipeline B (Runs only if A succeeds)
├── Pipeline C (Runs in parallel)
└── Pipeline D (Runs only if C succeeds)

Re: Dependency and Parallel Processing

Anonymous — Thu, 06 Mar 2025 11:38:03 GMT

im looking for a design , if my above ask is possible or not in ms fabric

Re: Dependency and Parallel Processing

ahmetyilmaz — Thu, 06 Mar 2025 11:40:43 GMT

Hi, parallel pipelines within the same pipeline already work without being affected by each other. Even if one of them fails, the other pipelines will work, but the main pipeline will return a failure.

Re: Dependency and Parallel Processing

NandanHegde — Thu, 06 Mar 2025 11:42:09 GMT

A and C run in parallel

Re: Dependency and Parallel Processing

Anonymous — Thu, 06 Mar 2025 11:47:32 GMT

Any Dependencies should i consider for this

Re: Dependency and Parallel Processing

NandanHegde — Thu, 06 Mar 2025 12:03:22 GMT

Above is the dependecy flow wherein the 2 pipelines should be dependant via on Success