Getting data from Azure Event Grid backwards

[foks]

Hi,

I have Azure Event Grid Namespace. The retention period there is 7 days. However, I'm not able to get data from previous days with my evenstream. When I'm changing show results from: 7 days, I see nothing. So how is the retention period really working? Azure Event Grid has 7 days retention period. As far as I can see, the connection from Fabric to Azure Event Grid creates an "Push" subscription. But what if I would like to use "Queue" subscription? How I can connect there?

[tayloramy]

Hi @foks, 

The Fabric Eventstream will only recieve events when it is running. If you pause the capacity, it will no longer recieve events. 

Eventstreams are intended for near real time data movement, not for batch processing of historical data. 

If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.

[foks]

Sure, but I'm wondering how to handle bad scenarios for e.g. where there is a capacity/Fabric outtage and I'm syncing Evenstream with Lakehouse and don't want to lost any streamed data. 

[tayloramy]

Hi @foks, 

The retention means that Event Grid is keeping the published events, so you should be able to seem them in Azure, however an eventstream will only pull new events, event streams don't go back and get historical data. 

If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.

[foks]

Hi @tayloramy ,

Many Thanks for the answer.

Are you sure that the behaviour is the same for Azure Event Hubs (pull-based) and Azure Event Grid (push-based)? I'm wondering what will happen if I disable my Fabric capacity and send events using Azure Event Grid. Will I be able to get them after I launch my capacity again?

[kustortininja · Accepted solution]

Hi @foks - Eventstreams create an internal subscriber that monitors the last time pulled from the event grid. In the event of a capacity or eventstream pause, when it is resumed it will pull the events from the last time the events were processed. 

[foks]

Thanks. It makes sense now.