Data Pipeline Storage Event Triggers (Preview): Ignore Empty Files
A common scenario in Azure Data Factory (ADF) when creating storage event triggers, was to ignore empty files (blobs). In ADF this was easily achieved by setting the ignoreEmptyBlobs property to either true or false.
In Fabric we offer an extensive list of filters to precisely identify the event we want our data pipelines to trigger. However, this can be challenging without proper insights into the event grid schema for Microsoft.Storage.BlobCreated and other event types.
Good thing is that we provide example events on the Azure Event Grid learn docs found here and their schema definitions (event properties) found here.
The property contentLength is an integer value indicating the size of the blob in bytes.
While creating the storage event trigger, on the Configure event type and source, we can add a new filter to only trigger when data.contentLength is great or equal to some number of bytes.
Resources and References
Data pipelines storage event triggers in Data Factory (Preview) - Microsoft Fabric | Microsoft Learn
Create event-based triggers - Azure Data Factory & Azure Synapse Analytics | Microsoft Learn
Azure Blob Storage as Event Grid source - Azure Event Grid | Microsoft Learn
Conclusion
I hope this quick blog helps anyone is interested in leveraging storage event triggers in Fabric data pipelines find and leverage all the various event filters available. Thank you.