Copy Job - retry failed

[jj44]

Hi all,

I’m looking for some advice / shared experience around pipeline activity timeout and retry behaviour in Fabric.

I’ve got a number of pipelines with multiple activities, and I’ve previously run into issues where Copy Job activities appear to hang when left with the default timeout (0.12:00:00). To mitigate that, I’ve reduced timeouts significantly (typically 0.00:10:00) and configured retries (retry = 2, retry interval = 30 seconds).

This has generally improved matters, but I’ve recently seen the following error:

"CopyJob execution failed… A job instance of the same job type is already running and this job instance is skipped"

From what I can tell, the timeout/retry logic is working as configured, but it looks like the retry may be kicking in before the previous job has fully terminated in the backend.

In most cases, the Copy activity itself only takes 2–3 minutes to run, so a 10-minute timeout should be more than sufficient.

My questions are:

• Has anyone else run into this behaviour?
• Is it likely that the retry interval (30 seconds) is too short, and the previous job is still “cleaning up” when the retry starts?
• What timeout / retry / retry interval settings are people typically using for Copy activities?

I’m trying to find a sensible balance between failing fast and avoiding these overlapping job issues.

Any insight would be really appreciated.

Thanks Jeff

[deborshi_nag · Accepted solution]

Hello @jj44 

The Fabric Copy Job activity may have timed out, but it is probably still working in the background, processing cleaning-up activities. This processing may take more than 30 secs, so when the retry event kicks in, the concurrency guard stops it from running. 

It is best you give it enough time before retry starts, maybe 1 - 3 mins for small jobs and 3 - 8 minutes for heavy duty copy jobs.