Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Be one of the first to start using Fabric Databases. View on-demand sessions with database experts and the Microsoft product team to learn just how easy it is to get started. Watch now

Reply
bcdobbs
Super User
Super User

Email on Error in Pipeline

I've built a pipeline that runs two notebooks and I'm wanting it to email on failure.

It looks like:

bcdobbs_0-1701769113553.png

With the Send Error Email connected on the fail nodes.

However it never triggers the outlook task:

bcdobbs_1-1701769174851.png

 

bcdobbs_2-1701769205113.png

Any suggestion on what I'm doing wrong or is there a better way to handle this?

 

Thanks

 

Ben



Ben Dobbs

LinkedIn | Twitter | Blog

Did I answer your question? Mark my post as a solution! This will help others on the forum!
Appreciate your Kudos!!
1 ACCEPTED SOLUTION
Anonymous
Not applicable

Yes @NandanHegde got it.

vgchennamsft_0-1701787396795.png


It looks like we don't have any issue with data factory, it is considered as a AND operator, not a OR operator.

@bcdobbs  I was able to trigger an email by creating a pipeline design like above.

View solution in original post

10 REPLIES 10
smpa01
Super User
Super User

@bcdobbs I belive this would be possible.

 

You need to declare a global variable which is an empty array.

Then you need to create an Append Variable activity that you would connect to each of the activities in the pipeline and you would keep on appending the succees/fail status in the append variable. At t-1 step, you would create a if condition to check if the collection (array) contains any fail using contains. If yes, send an email. I did not give it a try though. Let me know if it works.

Did I answer your question? Mark my post as a solution!
Proud to be a Super User!
My custom visualization projects
Plotting Live Sound: Viz1
Beautiful News:Viz1, Viz2, Viz3
Visual Capitalist: Working Hrs
Anonymous
Not applicable

The cleaner solution would be to wrap your pipeline within another pipeline, using the Invoke Pipeline activity.

 

  1. Create your pipeline without any notification activities.
  2. Create a second empty pipeline, and add Invoke Pipeline to call your first pipeline.
  3. Add an Outlook activity to the "On fail" node of the Invoke Pipeline activity.

This will trigger the email if any part of your inner pipeline fails.

S_Rahlf
Regular Visitor

I was checking that out myself before stumbling upon this community thread as it is "an old friend from data factory" -> all "inbound" constraints are combined AND so a mulitple "on failure" from different activities will never run.

Our workaround, as a "sequence container" is also not available, is to use a sub pipeline returning in case of a failure in any activity a "on failure" from the "invoke pipeline" activity or as kind of "dirty hack" an "if condition" as sequence having a "always true" condition and grouping activites...

S_Rahlf_0-1713455713978.png

 

NandanHegde
Super User
Super User

Hey,

You right fully identified the cause that the email would be triggered only if both the notebook activities would fail as the expressions are AND scenario and that would never occur as the 2nd notebook would be trigerred only with the success of the 1st one.

 

So as a solution, you can have seperate notifications per activity but there is only a limitaion of 40 activities per pipeline.

So below blog though ADF related would have the same concpets for Fabric:

https://datasharkx.wordpress.com/2021/08/19/error-logging-and-the-art-of-avoiding-redundant-activiti...

which would be a cleaner way




----------------------------------------------------------------------------------------------
Nandan Hegde (MSFT Data MVP)
LinkedIn Profile : www.linkedin.com/in/nandan-hegde-4a195a66
GitHUB Profile : https://github.com/NandanHegde15
Twitter Profile : @nandan_hegde15
MSFT MVP Profile : https://mvp.microsoft.com/en-US/MVP/profile/8977819f-95fb-ed11-8f6d-000d3a560942
Topmate : https://topmate.io/nandan_hegde
Blog :https://datasharkx.wordpress.com
Anonymous
Not applicable

Yes @NandanHegde got it.

vgchennamsft_0-1701787396795.png


It looks like we don't have any issue with data factory, it is considered as a AND operator, not a OR operator.

@bcdobbs  I was able to trigger an email by creating a pipeline design like above.

Anonymous
Not applicable

Hi @bcdobbs ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered. Otherwise, will respond back with the more details and we will try to help .

Anonymous
Not applicable

Hi @bcdobbs ,

We haven’t heard from you on the last response and was just checking back to see if your query was answered. Otherwise, will respond back with the more details and we will try to help .

hey @bcdobbs 

Are there any queries w.r.t creating a flow within the pipelines ?




----------------------------------------------------------------------------------------------
Nandan Hegde (MSFT Data MVP)
LinkedIn Profile : www.linkedin.com/in/nandan-hegde-4a195a66
GitHUB Profile : https://github.com/NandanHegde15
Twitter Profile : @nandan_hegde15
MSFT MVP Profile : https://mvp.microsoft.com/en-US/MVP/profile/8977819f-95fb-ed11-8f6d-000d3a560942
Topmate : https://topmate.io/nandan_hegde
Blog :https://datasharkx.wordpress.com
Anonymous
Not applicable

Hi @bcdobbs ,

Thanks for using Fabric Community.

As I tried to repro your scenario I have noticed this.

Case 1: Every thing worked well

vgchennamsft_2-1701771532862.png



Case 2: I can find the same issue you are facing

vgchennamsft_3-1701771666679.png

For time being try to link to only one Notebook Activity on failure.

In meantime I will check with internal to get some information on this behaviour.

Thank you

 

 

bcdobbs
Super User
Super User

Having looked at the JSON for the pipeline I now realise the way I'd built it would only trigger an error if both notebooks failed which couldn't happen in that setup.

I've added separate notification blocks and it now works.

bcdobbs_0-1701770214536.png

Is there however a cleaner way of doing this? Eg trigger an email on any failure anywhere in the pipeline?



Ben Dobbs

LinkedIn | Twitter | Blog

Did I answer your question? Mark my post as a solution! This will help others on the forum!
Appreciate your Kudos!!

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

November Carousel

Fabric Community Update - November 2024

Find out what's new and trending in the Fabric Community.

Dec Fabric Community Survey

We want your feedback!

Your insights matter. That’s why we created a quick survey to learn about your experience finding answers to technical questions.