Understanding Timeout in Activities

[HamidBee]

Hi all,

I'm trying to get a clearer understanding of the Timeout setting in Fabric activities. My current understanding is that Timeout determines how long an activity is allowed to run before it stops. So, for example, if I set a Timeout of 48 hours on a Copy Data activity that only needs 3 minutes to complete, would that limit how much data gets copied within the allowed time, or does it simply allow the activity up to 48 hours to finish?

Additionally, I'm wondering why setting a Timeout is necessary. Since activities in a pipeline generally move on to the next step once completed, what is the purpose of specifying a timeout duration? Are there specific scenarios where this functionality becomes crucial?

Thanks in advance for your insights.

[jpelham · Accepted solution]

If you have a copy data activity that only typically takes 3 minutes to complete, I would recommend not allowing itto run for 48 hours. You should be setting the timeout to a time that allows for the activity to reasonably complete with a small buffer. The minimum timeout is 10 minutes. If there ends of being a connection issue and the activity is allowed to run for 48 hours when it typically only takes 3 minutes, you will use more resources than necessary. 

Instead of using long periods of time before an activity times out, you should use retries, and also maybe incorporate sending emails or teams messages if an activity fails. For example, if you set a timeout of 10 minutes and have retries set to 2, your activity will attempt to complete up to 3 times and run for up to 30 minutes. If it still fails to load, then you could set up notifications on failure. This allows you catch potential issues with the pipeline, data source, or the gateway. 

[HamidBee]

Thanks for the explanation.

[v-kongfanf-msft]

Hi @HamidBee ,

Based on your confusion, here is my answer:

• This setting does not limit the amount of data that can be replicated in the allowed time, but allows the activity to run for a maximum of 48 hours. The Timeout is set to prevent the activity from running indefinitely, especially in the event of problems or long periods of incompletion.

• Specific scenarios: For example, when working with large data sets or complex computational tasks, the activity may take a long time to complete. By setting an appropriate Timeout, you can ensure that the activity completes in a reasonable amount of time, avoiding the system resource usage that can result from long running times.

For more details, you can refer to below document:

Activity overview - Microsoft Fabric | Microsoft Learn

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.