topic Re: Triggers of Fabric Pipelines-Concurrency in Pipelines

Triggers of Fabric Pipelines-Concurrency

Anonymous — Tue, 18 Jun 2024 07:41:39 GMT

Hi Team,

We have a requirement as described below.

There is a pipeline which will be triggered based on a file arrival in ADLS Container. When a file-1 is arrived and trigger starts the pipeline , there is possibility of file-2 arriving when first one is still in progress. The file-2 also should also trigger the same pipeline. Is there possibility of both of them being run parallely in fabric?Also please explain the option of concurrency and what are it's limitations.

Please provide your insights, it would be very helpful in deciding our solution.

Re: Triggers of Fabric Pipelines-Concurrency

NandanHegde — Tue, 18 Jun 2024 08:19:24 GMT

Under Pipeline settings,

there is a concurrency option.

By default, there is no limitation on cincurrent runs but you can define what can be the max concurrent runs.

In case if no concurrency is set up, based on Event trigger of 'N' number of files, there would be 'N' number of parallel executions.

Re: Triggers of Fabric Pipelines-Concurrency

Anonymous — Tue, 18 Jun 2024 09:20:06 GMT

Hi Anonymous ,

Thanks for using Fabric Community.
As NandanHegde mentioned you can configue the concurrent settings in pipeline in order to make 'N' number of parallel executions.

Just want add one more point here, limitations to concurrent operations -

Docs to refer - Data Factory limitations overview - Microsoft Fabric | Microsoft Learn

Hope this is helpful. Please do let me know incase of further queries.

Re: Triggers of Fabric Pipelines-Concurrency

Anonymous — Wed, 19 Jun 2024 06:07:00 GMT

Hi NandanHegde ,

Thanks for the inputs, they are very helpful.

Re: Triggers of Fabric Pipelines-Concurrency

Anonymous — Wed, 19 Jun 2024 06:08:00 GMT

Hi Anonymous ,

Thanks for the info, currently we are testing this feature on our pipelines, will get back to you if any further help is required.

Re: Triggers of Fabric Pipelines-Concurrency

Anonymous — Fri, 21 Jun 2024 10:01:26 GMT

Hi Anonymous ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered.
Otherwise, will respond back with the more details and we will try to help .

Thanks

Re: Triggers of Fabric Pipelines-Concurrency

YevgenyM — Thu, 22 Jan 2026 19:33:12 GMT

Is there a way to set pipeline concurrency to 1 by default?