Issue with using the external call activity more than once in the same data flow

[UmairSyed]

I am facing timeouts when I try to use the external call activity more then once in a data flow. Basically the first activity needs to make an API call to an external endpoint and based on the response, I need to make an update to another endpoint. Is there a limitation that this activity cant be used for more then once since all the actions after the first instance will be processed again for each row of data anyways?

[Anonymous · Accepted solution]

Hi @UmairSyed ,

Here are my personal thoughts on your question:

1. Each workspace has a limit on concurrent pipeline runs and activities. If you're hitting these limits, it could cause timeouts.

2. The external endpoints you're calling might have rate limits that restrict the number of requests you can make in a given time period. If you're making multiple calls in quick succession, you might be exceeding these limits.

3. If your data flow processes a large number of rows, each triggering an external call, it can lead to performance issues and timeouts.

To mitigate these issues, you might consider:

1. Instead of making an external call for each row, batch your requests to reduce the number of API calls.

2. Adjust the timeout settings for your activities to allow more time for the external calls to complete.

3. You can read this document for a further study: 

Data Factory limitations overview - Microsoft Fabric | Microsoft Learn

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.