Forum Discussion
Trigger Alerts Notification for successful/failed Pipelines
Hi Anonymous
To address your scenario in Microsoft Fabric, where you have a master pipeline with multiple interconnected child pipelines and want to trigger email notifications upon success or failure, here are some potential solutions:
Using Fabric API with a Notebook Activity
Your idea of using a single notebook activity to parameterize pipelines and capture logs is a good approach. Here's how you can implement it:
Use the Fabric API to monitor pipeline runs programmatically.
Create a notebook activity in your pipeline that:
Fetches the pipeline run status using the Fabric API.
Logs the status (success/failure) for each pipeline and its child pipelines.
Sends an email notification using an SMTP library (e.g., Python's smtplib) or a third-party email service like SendGrid.
Parameterize the notebook to handle different pipelines dynamically.
Using Webhooks for Notifications
You can use webhooks to trigger notifications based on pipeline events.
Configure a webhook in Microsoft Fabric to listen for pipeline run events (success/failure).
Use a service like Azure Logic Apps, Power Automate, or a custom webhook handler to process the event and send email notifications.
Ensure the webhook is triggered for both the master and child pipelines.
Custom Logging and Notification Pipeline
Create a separate pipeline dedicated to logging and notifications.
At the end of each pipeline (master and child), add an activity to log the run status to a centralized location (e.g., Azure Blob Storage, SQL Database).
Trigger the logging and notification pipeline after the master pipeline completes.
The logging pipeline reads the run statuses and sends email notifications.
Using Azure Monitor Alerts
Leverage Azure Monitor to track pipeline runs and send alerts.
Enable diagnostic settings for your pipelines to send logs to Azure Monitor.
Create an alert rule in Azure Monitor to trigger on specific conditions (e.g., pipeline failure).
Configure the alert to send email notifications.
Combining Fabric API and Logic Apps
If you want a more robust solution, combine the Fabric API with Azure Logic Apps.
Use the Fabric API to fetch pipeline run statuses.
Trigger a Logic App workflow to process the statuses and send email notifications.
Schedule the Logic App to run periodically or trigger it from the pipeline.
Regards,
Nono Chen
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hi Anonymous ,
Thank you for the logic, I happen to use notebook at pipeline run time by using a wait activity(as the status might take some time to reflect since we are using API). I happen to use the notebook during pipeline run where I fetch my latest run. I am able to extract this. However, I am trying to use Outlook activity to push the notification. The issue is I am not sure how to convert my string output to Table.
This is the structure of my notebook df output:
| Main Pipeline Name | Master Pipeline Name | Pipeline Name | Invoke Pipeline Name | Pipeline ID | Status | Error Message |
| PL_Parameter_ | Main Pipeline | Invoke pipeline1 | PL_Parameter_ | xyz-abc | Failed | {Error_Message_will_be_displayed} |
| PL_Parameter_ | PL_Parameter_ | Invoke pipeline1 | PL_Test_1 | efg-fhi | Failed | {Error_Message_will_be_displayed} |
I am using mssparkutils to capture this notebook output(exitValue) as a string and pass it to set variable(optional) then to Outlook activity. But since it is string, I am not sure how I can convert this string to table.
Also, if we want to use SMTP, can I use FabricRestClient instead of passing username and password.
Can you please share a sample structure for SMTP code as I don't want to pass username and password.
Thanks in advance!
Hrishi
Also, we are only looking to integrate this as a complete Fabric solution (we don't want to use Azure).
v-saisrao-msftCommunity Support
Hi Anonymous
Thanks for the detailed context you're on the right track with using a notebook and Fabric APIs. To convert your DataFrame output into a format suitable for the Outlook activity, I recommend using df.to_html(index=False) within your notebook to transform the DataFrame into an HTML table string. You can then pass this HTML string using mssparkutils.notebook.exit(), capture it in a Set Variable activity, and use it as the email body in your Outlook activity (assuming it supports HTML content). This approach maintains formatting and ensures readability in email notifications. Regarding SMTP, FabricRestClient currently doesn’t support credential-free SMTP directly. If avoiding username/password is critical, consider using the Outlook activity (if supported in your tenant) or triggering a webhook to Power Automate for notifications, which provides secure and flexible email delivery without handling credentials directly.
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.
Thank you.