Logging and tracking of alaram events for post-analysis and system improvementsI'm working with Microsoft Fabric's Data Activator to monitor real-time device temperatures. My goal is to implement a solution that does the following whenever a temperature exceeds a predefined threshold:Trigger an Alert:Send an email notification to the relevant stakeholder when the threshold is crossed.Log Alarm Data:Write the event into two separate Lakehouse tables:alarm_event_log: stores the actual telemetry data that caused the alarm.alarm_metadata: stores metadata about the alarm (timestamp, severity etc.).Execute Logging via Notebook:Use the "Action" section of the Data Activator rule to trigger a Fabric notebook that logs this data into the Lakehouse. (in action i am using fabric item which is notebook)What I need help with:How can I retrieve the data from Data Activator in the triggered notebook?Specifically, how do we access the relevant data (e.g., device ID, temperature reading, time of alert) that triggered the rule, and then pass or retrieve it within the notebook to log it properly?so like is there any way to retrieve that particular record from activator which caused the alert, if yes then how to implement it, I am planning to write the pyspark script for it, but little confuse about the logicWhat are the best practices or recommended methods for passing parameters from Data Activator to a Fabric notebook?Has anyone implemented a similar pattern?Any sample implementation or guidance on structuring this kind of workflow for reliability and scalability would be very helpful.Thanks in advance for your support!

Hi ramankr48 , Thanks for reaching out to the Microsoft fabric community forum. You're right , when you choose a Notebook as the action in Data Activator, it doesn't show a separate "Input Fields" section like it does for Power Automate. But no worries, you can still pass data to the notebook using the "Edit action" option. Here's what to do: Click on "Edit action" under the notebook you selected. In the Parameters section, paste something like this:{ "deviceId": "{{deviceId}}", "temperature": "{{temperature}}", "timestamp": "{{timestamp}}", "severity": "High", "ruleName": "TempThreshold"} These placeholders ({{deviceId}}, etc.) will automatically be filled in by Data Activator when the alert is triggered. In your notebook (PySpark), you can access them like this: device_id = parameters.get("deviceId") temperature = parameters.get("temperature") So even though the input fields aren’t visible upfront, this method works perfectly for passing alert data into your notebook. Thank you, Tejaswi.

how to log the data which is triggering alert in data acyivator

8 Replies

Anonymous
Not applicable
1 year ago
Hi ramankr48 ,

Thanks for reaching out to the Microsoft fabric community forum.

Set up Data Activator Rule

In Microsoft Fabric > Data Activator, create a rule with a condition:
e.g - temperature > 70

In the Action section:

Select Notebook as the Fabric item.

Add parameters to send with the alert. Example:
{
"deviceId": "{{deviceId}}",
"temperature": "{{temperature}}",
"timestamp": "{{timestamp}}",
"severity": "High",
"ruleName": "TempThreshold"
}

In Your Notebook (PySpark)

You'll now receive these parameters via the notebook's parameters dictionary. Here's how to access them and log the event:

# Access parameters passed from Data Activator
device_id = parameters.get("deviceId")
temperature = float(parameters.get("temperature"))
event_time = parameters.get("timestamp")
severity = parameters.get("severity")
rule_name = parameters.get("ruleName")

Create DataFrames for Logging

alarm_event_log (raw event)

from pyspark.sql import SparkSession
from datetime import datetime

event_log_df = spark.createDataFrame([{
    "device_id": device_id,
    "temperature": temperature,
    "event_time": event_time
}])

alarm_metadata (logging info)

metadata_df = spark.createDataFrame([{
    "device_id": device_id,
    "event_time": event_time,
    "logged_time": datetime.now().isoformat(),
    "severity": severity,
    "rule_name": rule_name
}])

Write to Lakehouse Tables

Make sure the Lakehouse tables alarm_event_log and alarm_metadata already exist or create them with schema accordingly.

# Append to Lakehouse tables -
event_log_df.write.mode("append").saveAsTable("YourLakehouse.alarm_event_log")
metadata_df.write.mode("append").saveAsTable("YourLakehouse.alarm_metadata")

If the response has addressed your query, please Accept it as a solution and give a 'Kudos' so other members can easily find it.

Best Regards,
Tejaswi.
Community Support
- ramankr48
  Helper II
  1 year ago
  like in the input fields, i need to add this part
  
  {
  "deviceId": "{{deviceId}}",
  "temperature": "{{temperature}}",
  "timestamp": "{{timestamp}}",
  "severity": "High",
  "ruleName": "TempThreshold"
  }
  
  in the same format or just name will also work
  - Anonymous
    Not applicable
    1 year ago
    Hi ramankr48 ,
    
    Thanks for reaching out to the Microsoft fabric community forum.
    
    Yes, you don’t need to add the full JSON structure in the Input Fields section.
    
    Just type in the names of the fields you want to pass, like:
    
    sqlCopyEditdeviceId
    temperature timestamp
    severity
    ruleName
    
    That’s enough, Data Activator will automatically fill in the values from the event that triggered the alert.
    
    Then in your notebook, you can access those values using the parameters.get("fieldName") method, like this:
    
    device_id = parameters.get("deviceId")
    
    So no need to write the whole JSON there, just the field names will work fine.
    
    If the response has addressed your query, please Accept it as a solution and give a'Kudos' so other members can easily find it.
    
    Best Regards,
    Tejaswi.
    Community Support
- ramankr48
  Helper II
  1 year ago
  so when you choose a fabric item in action part there is nmp option for input field
  - Anonymous
    Not applicable
    1 year ago
    Hi ramankr48 ,
    
    Thanks for reaching out to the Microsoft fabric community forum.
    
    You're right , when you choose a Notebook as the action in Data Activator, it doesn't show a separate "Input Fields" section like it does for Power Automate.
    
    But no worries, you can still pass data to the notebook using the "Edit action" option.
    
    Here's what to do:
    
    Click on "Edit action" under the notebook you selected.
    
    In the Parameters section, paste something like this:
    {
    "deviceId": "{{deviceId}}",
    "temperature": "{{temperature}}",
    "timestamp": "{{timestamp}}",
    "severity": "High",
    "ruleName": "TempThreshold"
    }
    
    These placeholders ({{deviceId}}, etc.) will automatically be filled in by Data Activator when the alert is triggered.
    
    In your notebook (PySpark), you can access them like this:
    
    device_id = parameters.get("deviceId")
    
    temperature = parameters.get("temperature")
    
    So even though the input fields aren’t visible upfront, this method works perfectly for passing alert data into your notebook.
    
    Thank you,
    
    Tejaswi.

Forum Discussion

how to log the data which is triggering alert in data acyivator

Here's what to do:

In your notebook (PySpark), you can access them like this:

8 Replies

Set up Data Activator Rule

In Your Notebook (PySpark)

alarm_event_log (raw event)

alarm_metadata (logging info)

Write to Lakehouse Tables

Here's what to do:

In your notebook (PySpark), you can access them like this:

Related content

Storage trigger event

How to Trigger pipeline using Event trigger

Pipeline Trigger Type

Unable to Trigger Fabric pipeline

Trigger Automation