Multiple pipeline runs triggered from single file creation in OneLake using Event Trigger

Hi MuhammedLabeeb,

single file creation triggers multiple pipeline runs is a known behavior in Microsoft Fabric and Azure Data Factory (ADF) when using storage event triggers. This phenomenon is often due to how Azure Blob Storage and Event Grid handle file creation events.

When a file is uploaded to Azure Data Lake Storage Gen2, especially using tools like Azure Storage Explorer, AzCopy, or programmatic methods, the upload process may involve multiple 

steps please refer this doc :https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-event-trigger?tabs=data-factory

• Chunked Uploads: Large files are often uploaded in chunks. Each chunk can emit its own event.
• Metadata Updates: After the file is fully uploaded, additional events like BlobPropertiesUpdated or BlobMetadataUpdated may be triggered. These multiple events can cause the event trigger to fire multiple times for what appears to be a single file creation.

   

   

  Thanks,

  Prashanth Are

  MS Fabric community support

Hi MuhammedLabeeb,

Specifying a “blob path” with just a file extension near the root may cause an unexpected spike in triggered pipelines. Ensure that your trigger definitions are precise and not too broad, as this can lead to multiple triggers being fired

Can you please refer to below doc related Event-Driven Architecture Through Storage Event Trigger and let me know if this helps resolve your issue?
microsofteur.sharepoint.com/teams/MissionCriticalReadiness/Shared Documents/Forms/AllItems.aspx?id=%2Fteams%2FMissionCriticalReadiness%2FShared Documents%2F3rd edition - FY24%2FSession 01-Azure Data Factory %26 Azure Synapse Analytics%2FEvent-Driven Architecture Through Storage Event Triggers%2Epdf&parent=%2Fteams%2FMissionCriticalReadiness%2FShared Documents%2F3rd edition - FY24%2FSession 01-Azure Data Factory %26 Azure Synapse Analytics

Thanks,

Prashanth Are