Multiple Pipeline run at the same time when hit the "run" button manually

[stellahe102]

When manually run the pipeline in fabric, by hitting the "run" button (see below screenshot1), there are multiple jobs are kicked off at the same time (see below screenshot2), which causes issue that the destination table in copy activity is loaded multiple times. The concurrency in the pipeline is set to 1.

Any idea how to make the pipeline only run once when manually run it by hitting the "run" button?

[stellahe102]

Thanks for the help!

[v-tsaipranay]

Hello @stellahe102 ,

Could you please confirm if the issue has been resolved after raising a support case? If a solution has been found, it would be greatly appreciated if you could share your insights with the community. This would be helpful for other members who may encounter similar issues.

Thank you for your understanding and assistance.

[v-tsaipranay]

Hello @stellahe102 

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[stellahe102]

No perfect solution found - but I just schedule the pipeline to run in 2 minutes during testing. as the scheduled job seems always just run once.

[v-tsaipranay]

Hi @stellahe102 ,

Could you please confirm if the issue has been resolved through the support ticket with Microsoft?

If the issue has been resolved, we kindly request you to share the resolution or key insights here to help others in the community. If we don’t hear back, we’ll go ahead and close this thread.

Should you need further assistance in the future, we encourage you to reach out via the Microsoft Fabric Community Forum and create a new thread. We’ll be happy to help.

Thank you.

[uselessai_in]

Hi, thanks for the question. Find all deployment related issues with their creative solutions here. I tested and implemented all of them. Hope it helps.

link - https://uselessai.in/dont-use-set-variable-activity-in-data-factory-pipeline-microsoft-fabric-bf74fa...

2. https://uselessai.in/microsoft-fabric-interesting-and-hidden-deployment-observations-important-bug-f...

3. https://uselessai.in/microsoft-fabric-stored-procedure-not-reflecting-warehouse-connection-change-in...

4. https://uselessai.in/microsoft-fabric-warehouse-deployment-issue-s-and-potential-solution-s-9ad36041...

[v-tsaipranay]

Hi @stellahe102,

Thank you for reaching out to the Microsoft fabric community forum regarding the issue you’re experiencing with your pipeline execution in Microsoft Fabric.

I understand that you are experiencing multiple jobs being initiated at the same time when you manually run the pipeline, even though the concurrency limit is set to 1. Please look into the following troubleshooting steps to resolve the issue:

• First, please ensure that there are no other triggers configured for your pipeline that might be executing it concurrently with your manual run. You can review and manage these triggers in the Pipelines section of the Fabric interface. If you find any additional triggers that could be causing this, consider disabling them temporarily to see if it resolves the issue.
• Need to make sure the pipeline isn’t accidentally set up to run multiple times. Can you check for any loops or parallel activities that might be causing this? Also ensure the activities are properly sequenced to prevent simultaneous runs.

I hope my suggestions give you good ideas, if you need any further assistance, feel free to reach out.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you. 

[stellahe102]

Hi @v-tsaipranay 

• I don't find any place to check if there is any trigger, and think there shouldn't be any one set up, as see below screenshot, there is only "Add Trigger (preview)" in my pipeline (vs "Trigger")
• there is a for each activity, but sequential step is already checked, so this reason should be ruled out too.

What I found out is, if I let the schedule to run the pipeline (like set it up to run in 2 minutes, and put the end date to today) then it's running correctly, but if I run it manually by hitting the run button, it's having multiple runs at the same time, any insight you might have based on this observation? or any further suggestions based on above 2 bullets points?

[v-tsaipranay · Accepted solution]

Hello @stellahe102 ,

We really apologize for the delay in response, try verify if there are any existing instances of the pipeline running before starting a new manual run. If multiple instances are running, it may be beneficial to implement a check to prevent starting a new run if one is already in progress.

If this does not resolve your issue, I recommend raising a support ticket so they can assist in addressing the problem, please refer below link on how to raise a contact support or support ticket.

https://learn.microsoft.com/en-us/power-bi/support/create-support-ticket

Thank you.