How Useful is the Retry Setting in Fabric Pipelines?

[HamidBee]

Hi everyone,

I’m curious about the Retry option for activities in Fabric pipelines. My initial understanding was that if an activity fails, it usually indicates an underlying issue in the pipeline, such as a configuration or setting error. In these cases, simply retrying wouldn't solve the problem unless those issues are addressed.

How beneficial is it to set retries for an activity? Are there specific scenarios where retries actually help ensure success, or is this setting more useful in certain types of tasks? Would love to hear your thoughts and experiences with this setting.

Thanks

[jpelham]

I have found retries to be most beneficial for runnig notebooks. It is common for a notebook to fail on the first one or two attempts. Occassionally there is a glitch or a timeout that affects the gateway as well.

I would suggest testing a newly created pipeline activity with retry set to 1 to account possible such issues, while also making sure the activity is calibrated correctly. Once you are sure the activity is set up properly, I would recommend setting the retry attempts to 2 or 3 depending on the activity. In general, I set notebooks to 3 with a 30 sec interval, and other activities to 2.

[Anonymous · Accepted solution]

Hi @HamidBee ,

First of all I think you're right that if an activity fails due to a configuration or setup error, simply retrying it won't solve the underlying problem.

However, setting up retries for activities in the fabric pipeline can still be beneficial in several situations:

1. Sometimes, failures occur due to temporary issues like network glitches, service unavailability, or timeouts. In such cases, a retry can help the activity succeed without manual intervention.

2. If the failure is due to resource contention (e.g., database locks or high CPU usage), a retry after a short delay might succeed when the resources become available.

3. Activities that depend on external systems or services might fail if those systems are temporarily down. Retries can help in such cases by giving those systems time to recover.

4. In distributed systems, retries can help balance the load by attempting the activity on a different node or server. 

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[HamidBee]

Thanks for sharing

[Srisakthi]

Hi @HamidBee ,

Basically Retry option helps in scenario where source system connection failure happens or or datagateway connection timeout due to system unavailable(or on-prem system might be in rebooting state) or  source files arrives few seconds later after the pipeline triggered. Happy to hear more scenarios than these.

Regards,

Srisakthi