Get an email if any of activities fail

[JānisB]

We have a pipeline with a lot of Copy activities. Each activity is connected "On completion". If the last activity doesn't fail then the whole pipeline status shows as Succeeded.

Our goal is to get an email if any of activities fail. It is possible to connect "On fail" another activity Office Outlook, but it is necessary to make this kind of acitivity for each Copy data activity (it is a problem, if there are a 100 tables).

Sometimes we make a pipeline for each table separately and then we have the main pipeline with "Invoke Pipeline" activities for each table pipeline. The same problem, that the only way to get fail email is to configure Office Outlook acitivity for each activity (table pipeline can contain for example 6 activities).

How is it possible to send an email if any of activities fail in convenient way?

[NandanHegde]

please refer the below blog for effective management of failure notifications :

https://datasharkx.wordpress.com/2021/08/19/error-logging-and-the-art-of-avoiding-redundant-activiti...

[v-hashadapu · Accepted solution]

Hi @JānisB , Thank you for reaching out to the Microsoft Community Forum.

The best way to do this is to structure your pipeline using centralized failure tracking with a ForEach container. Organize your Copy activities (or Invoke Pipeline calls for nested pipelines) inside a ForEach activity that loops over your table list. Connect all activities using "On Completion", so the pipeline doesn't stop on the first failure. If any iteration fails, the ForEach activity will fail and you can attach a single "On Failure" path from the ForEach to an Office Outlook activity (or a Logic App/Web activity) to send an email notification. This means one email activity handles failures across all steps, even with 100+ tables.

For nested pipelines, each child pipeline should manage its own internal failures. Inside the child pipeline, use a variable that gets set if any step fails. At the end, set an output parameter based on that variable. Back in the main pipeline, the ForEach can pick up this output and if any child pipeline fails, the ForEach will fail, again triggering the centralized email.

If this helped solve the issue, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details, always happy to help.
Thank you.