March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Early bird discount ends December 31.
Register NowBe one of the first to start using Fabric Databases. View on-demand sessions with database experts and the Microsoft product team to learn just how easy it is to get started. Watch now
I've built a pipeline that runs two notebooks and I'm wanting it to email on failure.
It looks like:
With the Send Error Email connected on the fail nodes.
However it never triggers the outlook task:
Any suggestion on what I'm doing wrong or is there a better way to handle this?
Thanks
Ben
Solved! Go to Solution.
Yes @NandanHegde got it.
It looks like we don't have any issue with data factory, it is considered as a AND operator, not a OR operator.
@bcdobbs I was able to trigger an email by creating a pipeline design like above.
@bcdobbs I belive this would be possible.
You need to declare a global variable which is an empty array.
Then you need to create an Append Variable activity that you would connect to each of the activities in the pipeline and you would keep on appending the succees/fail status in the append variable. At t-1 step, you would create a if condition to check if the collection (array) contains any fail using contains. If yes, send an email. I did not give it a try though. Let me know if it works.
The cleaner solution would be to wrap your pipeline within another pipeline, using the Invoke Pipeline activity.
This will trigger the email if any part of your inner pipeline fails.
I was checking that out myself before stumbling upon this community thread as it is "an old friend from data factory" -> all "inbound" constraints are combined AND so a mulitple "on failure" from different activities will never run.
Our workaround, as a "sequence container" is also not available, is to use a sub pipeline returning in case of a failure in any activity a "on failure" from the "invoke pipeline" activity or as kind of "dirty hack" an "if condition" as sequence having a "always true" condition and grouping activites...
Hey,
You right fully identified the cause that the email would be triggered only if both the notebook activities would fail as the expressions are AND scenario and that would never occur as the 2nd notebook would be trigerred only with the success of the 1st one.
So as a solution, you can have seperate notifications per activity but there is only a limitaion of 40 activities per pipeline.
So below blog though ADF related would have the same concpets for Fabric:
which would be a cleaner way
Yes @NandanHegde got it.
It looks like we don't have any issue with data factory, it is considered as a AND operator, not a OR operator.
@bcdobbs I was able to trigger an email by creating a pipeline design like above.
Hi @bcdobbs ,
We haven’t heard from you on the last response and was just checking back to see if your query was answered. Otherwise, will respond back with the more details and we will try to help .
Hi @bcdobbs ,
We haven’t heard from you on the last response and was just checking back to see if your query was answered. Otherwise, will respond back with the more details and we will try to help .
hey @bcdobbs
Are there any queries w.r.t creating a flow within the pipelines ?
Hi @bcdobbs ,
Thanks for using Fabric Community.
As I tried to repro your scenario I have noticed this.
Case 1: Every thing worked well
Case 2: I can find the same issue you are facing
For time being try to link to only one Notebook Activity on failure.
In meantime I will check with internal to get some information on this behaviour.
Thank you
Having looked at the JSON for the pipeline I now realise the way I'd built it would only trigger an error if both notebooks failed which couldn't happen in that setup.
I've added separate notification blocks and it now works.
Is there however a cleaner way of doing this? Eg trigger an email on any failure anywhere in the pipeline?
March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!
User | Count |
---|---|
6 | |
5 | |
3 | |
2 | |
1 |
User | Count |
---|---|
16 | |
13 | |
11 | |
7 | |
6 |