Check if a pipeline is currently running

[BW_RFA]

Hi,

I have a scheduled pipeline that runs every 15 minutes. 

Once a day it may ingest a lotof data and the pipeline runs for over 15 minutes, meaning the next run tries to process the same data etc.

Is there a way to automatically check whether the pipeline is currently running (from within the pipeline) so I don't run it multiple times if it currently ongoing? I would check to see if the pipeline is running and stop the current run if it is.

Thanks

[NandanHegde]

Unfortunately As of today Fabric data pipelines do not have any REST API because of which we cannot check the status of the pipeline at run time dynamically.

But you can enable the concurrency property wherein making it 1 would ensure only 1 instnace of pipeline is running and have others in queue.

[v-nikhilan-msft]

Hi @BW_RFA 
Thanks for using Fabric Community.
You can go to the monitoring hub in a pipeline to check the status of pipeline runs.

For more information please refer to this link:
https://learn.microsoft.com/en-us/fabric/data-factory/monitoring-hub-pipeline-runs

Hope this helps. Please let me know if you have any further queries.

[BW_RFA]

Hi,

Thank you for your reply. I think you have misunderstood my question.

I want to do this automatically from within a pipeline so that if the same pipeline is still running it will stop.

Thanks

[v-nikhilan-msft · Accepted solution]

Hi @BW_RFA 
Apologies for the misunderstanding. Directly checking the pipeline's status within an ongoing pipeline in Microsoft Fabric isn't currently available. You can set the concurrency limit to 1 in the pipeline settings. The maximum number of concurrent runs the pipeline can have is called Concurrency. By default, there is no maximum. If the concurrency limit is reached, additional pipeline runs are queued until earlier ones complete.

Hope this helps. Please let me know if you have any further queries.

[BW_RFA]

Thanks,

Does this work with scheduled triggers of the same pipeline?

[v-nikhilan-msft]

Hi @BW_RFA 
Yes the concurrency works for scheduled triggers also.