Forum Discussion
Multiple messages when new item created
I am attempting to create an alert when a new table is added to a lakehouse. I have created an eventstream from OneLake events, NewFileCreated and added an alert to send a Teams message when the event triggers, however we are receiving 6-20 messages per event. I am assuming this is creating an individual alert for each parquet file. Is there a way to only tirgger a single alert for each table?
Hi jmeisenh,
Thank you for the response, please follow these steps:
Go to the existing Eventstream, Make sure it is already connected to a OneLake data source and receiving NewFileCreated events. We need to extract the folder path from the full file path, this helps us group all files from the same table together.
- Add a Derived Column to extract just the folder path from the full file path, this represents the table.
- Add an Aggregate block, group by the folder path, add any simple aggregation just to trigger the alert. This will check for file events grouped by folder path in the preferred time period and we get only one event per table.
- Connect this to your Teams alert block.
So whenever a table is created, even if it creates more than 10 files, the Eventstream will send only one alert per table within the selected time window.
Please refer these below documents for your reference:
https://learn.microsoft.com/en-us/fabric/real-time-intelligence/tutorial-7-set-alert
https://learn.microsoft.com/en-us/fabric/real-time-hub/set-alerts-data-streams
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Thanks and regards,
Anjan Kumar Chippa
4 Replies
Hi jmeisenh,
Thank you for reaching out to Microsoft Fabric Community.
In Fabric, creating a new table generates multiple file creation events, so multiple alerts are expected if you directly use NewFileCreated from OneLake. Currently there is no built in way to get a single alert per logical table created from OneLake.
The best approach is to use sliding window aggregation in your Eventstream, grouping events by the table path and a short time window for example like 1 minute window. This ensures that only one alert is triggered per table, no matter how many files are created.
This is the best method for triggering one alert per table in Fabric.
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Thanks and regards,
Anjan Kumar Chippa
I guessed I would have to do something like this. My next question would be how? I am new to Real-Time Monitoring and could not find any documention on how to do this.
Hi jmeisenh,
Thank you for the response, please follow these steps:
Go to the existing Eventstream, Make sure it is already connected to a OneLake data source and receiving NewFileCreated events. We need to extract the folder path from the full file path, this helps us group all files from the same table together.
- Add a Derived Column to extract just the folder path from the full file path, this represents the table.
- Add an Aggregate block, group by the folder path, add any simple aggregation just to trigger the alert. This will check for file events grouped by folder path in the preferred time period and we get only one event per table.
- Connect this to your Teams alert block.
So whenever a table is created, even if it creates more than 10 files, the Eventstream will send only one alert per table within the selected time window.
Please refer these below documents for your reference:
https://learn.microsoft.com/en-us/fabric/real-time-intelligence/tutorial-7-set-alert
https://learn.microsoft.com/en-us/fabric/real-time-hub/set-alerts-data-streams
If this post helps, then please consider Accepting as solution to help the other members find it more quickly, don't forget to give a "Kudos" – I’d truly appreciate it!
Thanks and regards,
Anjan Kumar Chippa
