Forum Discussion
Eventstreams and Activator duplicate runs
Hi jajumonville andjjumonville,
Thank you for your feedback. I completely understand the importance of having a seamless and efficient solution, and I acknowledge the difficulties posed by the current preview functionality.
Since Fabric is still evolving, your insights are incredibly valuable in shaping its future. If you haven’t already, I encourage you to submit your feedback.
Microsoft Fabric Ideas Fabric Ideas - Microsoft Fabric Community, as this helps the product team prioritize improvements.
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.
Thank you.
unfortuanately i believe the problem lies with ADF not just writing the file at once but rather is generating unseen .tmp files both on creatation and deletion, with the event stream set to only pay attention to blob creation, it still triggers when then the blob is deleted. - This preview function does not appear to be suitable for production work yet. There needs to be control in the pipelines to prevent duplicate runs, this doesn't exist either
Hi jajumonville andjjumonville,
Thank you for identifying the core issue ADF generates temporary .tmp files during both creation and deletion. Fabric Eventstream, although configured to monitor blob creation, is triggering on both events. This lack of fine-grained filtering and state validation in Eventstream is causing the problem. We appreciate you highlighting this, as it helps us understand the limitations of the preview.
- Use a Success Marker File Instead of triggering on the CSV file, modify ADF to create a "done.signal" file only after successful completion. Eventstream can be configured to monitor only this marker file, preventing premature triggers.
- Modify ADF's Write Behavior If possible, adjust ADF to write to a temporary directory first, then move the final file to the monitored location once it's complete. This can help avoid detecting .tmp files.
Since it is still in preview, some of these limitations are expected, and improvements are actively being worked on. The full release will likely bring better event filtering options and built-in mechanisms to prevent duplicate pipeline executions.
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.
Thank you.
Hi jajumonville andjjumonville,
I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.
Thank you.This preview function is still too primitive to use without extreme amounts of workarounds, we are abandoning this and implementing a timed watch on a folder to check for the file, this is not ideal as 1 it eats up the amount of notebooks that can be called in a pipeline and 2 it just signals some things are ready for prime time yet, meaning we will need to slow our adoption of fabric down.
Hi jajumonville andjjumonville,
Thank you for your feedback. I completely understand the importance of having a seamless and efficient solution, and I acknowledge the difficulties posed by the current preview functionality.
Since Fabric is still evolving, your insights are incredibly valuable in shaping its future. If you haven’t already, I encourage you to submit your feedback.
Microsoft Fabric Ideas Fabric Ideas - Microsoft Fabric Community, as this helps the product team prioritize improvements.
If this post helps, then please give us ‘Kudos’ and consider Accept it as a solution to help the other members find it more quickly.
Thank you.
