Data Pipeline failure alert

[Srisakthi]

Hello Everyone,

I have tried adding MSteams activity when one of my activity fails. But is there anyway to alert when my pipeline fails?

Is there anyway to alert my data engineers when my pipeline fails?

Regards,

Srisakthi

[NandanHegde · Accepted solution]

Unfortunately, unlike ADF wherein you can have in built alerts to notify in case of pipeline failures; in fabric data pipelines you have to explictily manage the failure notifications yourself.

Rather than having team/outlook activity after each activity, you can use the below method :

https://www.c-sharpcorner.com/article/error-logging-and-the-art-of-avoiding-redundant-activities-in-...

[Srisakthi]

Hello,

I know that we can an activity to my pipeline and use OnFail action to take further actions by sending it to Outlook, teams or even i can use web activity to log it somewhere. Here the questions suppose i have 10 activity in my pipeline , do I need to add OnFail for each activity? I want to add only at one place and send a notification.

I found one way which is to add the respective pipeline in another pipeline(and use invoke pipeline) and attach to notification activity. Is there any other approach?

Regards,

Srisakthi

[SudhavaniKolla3]

you can add all activities and conenct them on success of first activity, second activity shoud run and on success of second activity, 3rd activity should run, like this you have to attach first, and at the end of last activity, you should add teams or outlook activity on failure of last activity.

so that if any activty fails then pipeline also gets fails, then you will get mail notification.

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[NandanHegde]

Plz refer this :

https://www.c-sharpcorner.com/article/error-logging-and-the-art-of-avoiding-redundant-activities-in-...

@SudhavaniKolla3 The above flow that you have created wont work in case if there is any failure in lookup or copy or dataflow activity. plz refer the above blog for the correct flow creation

[spencer_sa]

In addition to sending an Outlook message (as SudhavaniKolla3 says) you can also use the Teams activity to send a message to a Teams group or channel.

Be aware, using the OnFail flow anchors in pipeline activities is akin to a try: exception: block in that the main pipeline will no longer fail on that activity failing, so you'll need to add an explicit 'Fail' activity to force the main pipeline to fail.

[SudhavaniKolla3]

Hi, you can use Outlook 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.