Forum Discussion

jeanwang's avatar
jeanwang
New Member
29 days ago
Solved

how to set failure notification on the pipeline level

I have created a pipeline running multiple steps, if one step failed, i would like to move to next step till it finished the whole pipeline. I don't want to set a failure notification per step because that could end up sending out too many failure emails. I tried to create an activator for the pipeline job, the activator captured the failure and was able to send out a testing email, but it failed to send out the alert email in real environment. I wonder if there is another way to set up alert for the whole pipeline?

 

Thnx.

  • Hi jeanwang​,

    The Microsoft fabric support pipeline failure notification by default so there is no need to set up failure notification.

    Further in your case statement the recommended solution will be  

    Recommended Solution: 

    Activity 1 fails → continue Activity 2 → continue Activity 3 → finish the pipeline → send one email if the overall pipeline failed

    Use the pipeline-level Failure notification.

    Go to:

    Fabric Pipeline → Home → Schedule → Failure notifications

    Add the users/groups who should receive the notification. Microsoft specifically documents this as a pipeline-level notification for scheduled pipeline runThe pipeline should configure like this 

    Activity 1 ---> upon completion 
    Activity 2 ---> upon completion

    Activity 3 ---> upon completion

    So when pipeline moves to last activity, it will send failure notification, so you will get one notification instead of multiple notification.

    Upon completion means regardless of failure or success your pipeline will move to second activity.

     If this helps, ✓ Mark as Kudos | Help Others

8 Replies

  • Hi jeanwang

     

    I would approach this using variables. 

     

    Have a variable in your pipeline for "hasFailed" or something similar with a default value of 0, and at the end of every step, if it failed, set that variable to 1.  

    Then at the end of the pipeline, check the value of that variable, if it is 1, then send a failure email. 

    You can also add more variaqbles for FailureStep and FailureReason to give a more complete error message. 

  • v-abhinavmu's avatar
    v-abhinavmu
    Icon for Community Support rankCommunity Support

    Hi jeanwang​,

    I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. 


    Thank you.

  • Hi jeanwang​, there are many ways to approach your challenge that the community suggested above. Let me add a couple more:

    1. Run a monitoring pipeline or a job that calls Fabric REST API and checks statuses of recent pipeline runs and sends notifications for each found failure. This is somewhat similar to what you could see in Fabric monitoring hub only it allows you to implement your own custom failure handling. You can implement a REST API call in Python code in either a Python Notebook or a UDF and run them from a factory pipeline.
    2. Create a parent pipeline that first executes your own pipeline as a child activity and next executes a notification activity only if the child pipeline fails. This approach is simple and you can even parameterize the parent pipeline with a child pipeline ID so it runs any of your pipelines and sends a notification if a child pipeline fails. The parent pipeline will fully rely on the platform failure detection and won't care which activity in a child pipeline failed but you can use a pipeline activity output to build a custom error message for your notification.
  • v-abhinavmu's avatar
    v-abhinavmu
    Icon for Community Support rankCommunity Support

    Hi jeanwang​,

    May I check if this issue has been resolved? If not, Please feel free to contact us if you have any further questions.

    Thank you

  • Hi jeanwang​,

    The Microsoft fabric support pipeline failure notification by default so there is no need to set up failure notification.

    Further in your case statement the recommended solution will be  

    Recommended Solution: 

    Activity 1 fails → continue Activity 2 → continue Activity 3 → finish the pipeline → send one email if the overall pipeline failed

    Use the pipeline-level Failure notification.

    Go to:

    Fabric Pipeline → Home → Schedule → Failure notifications

    Add the users/groups who should receive the notification. Microsoft specifically documents this as a pipeline-level notification for scheduled pipeline runThe pipeline should configure like this 

    Activity 1 ---> upon completion 
    Activity 2 ---> upon completion

    Activity 3 ---> upon completion

    So when pipeline moves to last activity, it will send failure notification, so you will get one notification instead of multiple notification.

    Upon completion means regardless of failure or success your pipeline will move to second activity.

     If this helps, ✓ Mark as Kudos | Help Others

  • Kagiyama_yutaka's avatar
    Kagiyama_yutaka
    Icon for Continued Contributor rankContinued Contributor

    The platform only sends a failure alert when the pipeline run ends in Failed, so the easiest way to get a single alert is to add a final step that fails if any earlier step failed.

  • Hi jeanwang​,

    If these pipelines are running on a schedule, I would use the built-in pipeline failure notifications rather than adding an email activity after every step.

    In the pipeline, go to Home > Schedule, then add the users or groups under Failure notifications.

    That notification is tied to the overall pipeline run, so you get one failure notification when the scheduled pipeline fails rather than an email for each individual activity.

    The main thing to watch is your dependency logic. If you deliberately handle an activity failure and allow the rest of the pipeline to continue, make sure the overall pipeline still ends in a Failed state. Otherwise Fabric has no pipeline-level failure to notify on.

    If you need the pipeline to continue processing after an intermediate failure, I would usually keep track of whether anything failed and then use a Fail activity at the end when that flag is set. That way the remaining steps can run, but the final pipeline status is still Failed and the scheduled failure notification fires once.

    For manual or on-demand runs, Microsoft currently notes that these built-in failure notifications are not sent. In that case I would use a single Office 365 Outlook activity at the end instead, based on the same failure flag.

    AI-assisted drafting: AI was used to help structure and phrase this response. I reviewed and validated the technical content before posting.