Forum Discussion
event trigger authentication & error fetch
Dears,
We have an event trigger which I created from inside my fabric data factory pipeline by choosing the option onelake events (figure 1)
If I am not wrong, it works like this : onelake -> emits event -> Activator receivs it-> triggers the pipeline
In my case, I have configured it just for file creation, other 5 events I don't need them
Figure 1
1
Q1 : Do I need to add any type of security to the trigger (non-human security, like a service principal or a workspace identity)? I don't see it anyware in the UI, while creating the trigger.. but what does the activator uses to activate the pipeline? it runs in which security context? not sure how to secure this ..
Q2 : I have a table which receives metadata from the load runs, like : run_id, run_status, error_message, start_time, end_time, etc... which is fed by the notebooks that are called by this pipeline. This works very good. Every layer has notebbooks and this notebooks, when they run, they feed this table (one table per layer). What shall I do to log the events of : onelake -> emits event -> Activator receivs it-> triggers the pipeline ? this, to be pro-active, and understand in case something fails at the event level (not at pipeline level, as the pipeline logs the information as I mentioned before)?
Shall I get : onelake -> emits event -> Activator ? or only the activator information? My goal is to be aware in case something is not working properly with the event mechanism and so, we don't get the pipe triggered. Would you create a new table, just to log this mechanism (events) and if so, how to feed it?
Thanks a lot,
Pedro
Hi fabricpribeiro ,
A gap-detection check is a simple monitoring mechanism that ensures your event-driven pipeline is working correctly end to end. It works by checking whether a pipeline run happens within a defined time after a file is created in OneLake. If a file arrives but no corresponding pipeline execution is detected within that time window, it raises an alert (for example, via email or Teams). This helps you catch issues in the event-to-trigger process such as missed events or Activator failures even when your pipeline never starts and therefore doesn’t generate any logs.
Please check this article for more details:
Reacting to Azure Blob storage events | Microsoft LearnI hope this information helps. Please do let us know if you have any further queries.
Thank you
6 Replies
Hi fabricpribeiro ,
Thanks for reaching out to the Microsoft Fabric Community forum.At the moment, there is no need to configure a separate service principal or identity for the trigger within the UI. The trigger operates under the Fabric-managed identity within the workspace context, where the Activator service handles the event internally and executes the pipeline using the permissions associated with the workspace and the pipeline artifact.
For monitoring this flow, the recommended approach is to leverage Workspace Monitoring, specifically the Item Job Events. These provide visibility into when pipelines are triggered and executed, allowing you to track whether the event from OneLake successfully resulted in a pipeline run. While OneLake event emission and Activator-level logs are not directly exposed, Item Job Events act as the closest observable layer to validate the end-to-end trigger behavior. You can use these events to monitor execution patterns, detect missed triggers, and build alerting or logging mechanisms around pipeline runs without needing to create a separate custom logging solution for the event mechanism itself.
Reference: Workspace monitoring overview - Microsoft Fabric | Microsoft Learn
I hope this information helps. Please do let us know if you have any further queries.
Thank youhi fabricpribeiro ,
Q1) Security / identity
- No extra credential is configured in the trigger UI—security is controlled by Fabric permissions on the source item.
- To subscribe to OneLake events, the trigger owner must have SubscribeOneLakeEvents permission on the source (e.g., Lakehouse).
- If the owner loses that permission, the event subscription can go paused and stop delivering events (retained up to 7 days).
Recommendation: create/own the trigger with a stable non-human identity (SP/workspace identity) and grant only needed permissions.
https://learn.microsoft.com/en-us/fabric/real-time-hub/fabric-events-subscribe-permission
Q2) Logging the event mechanism (pre‑pipeline)
Persist OneLake events
- Capture raw OneLake file/folder events (for example: FileCreated, FileDeleted, FileRenamed, etc.) along with key fields like type, subject, and time.
- Stream these events into storage by creating an Eventstream from OneLake events and landing them into a Lakehouse/Eventhouse so you have a queryable history of “what happened in OneLake and when.”
Persist pipeline Job events
- Subscribe to Fabric Job events to track the pipeline lifecycle (job created / succeeded / failed) as an independent record that a run was actually initiated and how it ended.
Add a gap-detection check
- Build a simple gap check: when a FileCreated event arrives, expect a matching pipeline job created event within N minutes.
- If the job event doesn’t appear in time, raise an alert (Teams/email)—this flags failures in the event-to-trigger mechanism (even when your pipeline-level logging never starts)
https://learn.microsoft.com/en-us/fabric/real-time-intelligence/data-activator/activator-trigger-fabric-items
Optionally , you can also stamp your pipeline run metadata with trigger details using built-in trigger parameters like:
@pipeline()?.TriggerEvent?.FileName
https://learn.microsoft.com/en-us/fabric/data-factory/pipeline-runs
https://learn.microsoft.com/en-us/fabric/data-factory/pipeline-runs#event-based-pipeline-runs
Thanks,
ShreyaHello,
Can you please explain this part better ( I mean, with more verbose, please)?
Add a gap-detection check
- Build a simple gap check: when a FileCreated event arrives, expect a matching pipeline job created event within N minutes.
- If the job event doesn’t appear in time, raise an alert (Teams/email)—this flags failures in the event-to-trigger mechanism (even when your pipeline-level logging never starts)
Hi fabricpribeiro ,
A gap-detection check is a simple monitoring mechanism that ensures your event-driven pipeline is working correctly end to end. It works by checking whether a pipeline run happens within a defined time after a file is created in OneLake. If a file arrives but no corresponding pipeline execution is detected within that time window, it raises an alert (for example, via email or Teams). This helps you catch issues in the event-to-trigger process such as missed events or Activator failures even when your pipeline never starts and therefore doesn’t generate any logs.
Please check this article for more details:
Reacting to Azure Blob storage events | Microsoft LearnI hope this information helps. Please do let us know if you have any further queries.
Thank you
Hi fabricpribeiro
I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions.
Thank you.May I check if this issue has been resolved? If not, Please feel free to contact us if you have any further questions.
Thank you
