Forum Discussion
Event based trigger causing duplicate runs
I figured out a workaround for this. Instead of using File creation, I used file deletion event.
The deletion event doesnt cause duplicate runs! and I can still capture the information i need from the file name after deletion.
This also works as an automatic clean up activity as well haha 😁
Thank you svenchio , tayloramy , & v-kpoloju-msft for your valuable inputs! 🤝
Hi Karol_PL,
Thank you for reaching out to the Microsoft Fabric Community Forum.
You are observing this behaviour because a single logical file write in One Lake generates multiple FileCreated events, each representing a different internal operation. While the event type stays the same, the api field changes (for example, CreateFile when the write starts and FlushWithClose when the file is finalized). This is expected behavior in Fabric, and without additional filtering, it can result in duplicate pipeline triggers for the same file.
To prevent this, it does make sense to filter your trigger based on the api parameter. The recommended approach is to trigger the pipeline only when api = FlushWithClose, as this ensures the file is fully written and ready for processing. Adding this condition will allow the pipeline to run once per file and avoid duplicates, while still reliably capturing all completed file uploads.
Refer these links:
1. https://learn.microsoft.com/en-us/fabric/real-time-hub/tutorial-build-event-driven-data-pipelines 2. https://learn.microsoft.com/en-us/fabric/real-time-hub/explore-fabric-onelake-eventsThanks again for using the Microsoft Fabric Community Forum.
Hi Hamza_Amir,
Thank you for reaching out to the Microsoft Fabric Community Forum. Also, thanks to svenchio,tayloramy, for those inputs on this thread
Thank you for your patience and for taking the time to validate the notebook logic and share the exact code that really helps narrow things down. Based on everything you have tested, your implementation looks correct and you are not doing anything wrong in the notebook or in the Activator setup.
What you are observing is related to how Microsoft Fabric currently raises One Lake events. Internally, a single logical file write can go through multiple commit steps, and during that process the Microsoft.Fabric.OneLake.FileCreated event can be emitted more than once with identical event details. When this happens, Data Activator detects both events and triggers the pipeline twice, even though only one file was created from a user perspective.
Since this behaviour originates at the platform event level, the most reliable way to handle it is to make the downstream pipeline resilient to duplicate triggers. Common approaches include adding an idempotency check (for example, tracking processed file names/timestamps), validating file stability before processing, or introducing a short delay before execution. These patterns ensure that even if the event fires twice, the file is processed only once.
We understand this can be confusing when you expect a one-to-one mapping between file creation and trigger execution, and we appreciate you raising this scenario. Please let us know which approach fits your use case best, and we’ll be happy to help further.
Hope that clarifies. Let us know if you have any doubts regarding this. We will be happy to help.
Thank you for using the Microsoft Fabric Community Forum.
I figured out a workaround for this. Instead of using File creation, I used file deletion event.
The deletion event doesnt cause duplicate runs! and I can still capture the information i need from the file name after deletion.
This also works as an automatic clean up activity as well haha 😁
Thank you svenchio , tayloramy , & v-kpoloju-msft for your valuable inputs! 🤝