Add HTTP REST / Webhook source and destination to Eventstream

[svelde]

At this moment, Eventstream has no support for ingesting messages via HTTP REST / Webhook (like an Azure Function HTTP trigger).

The same goes for egress to an HTTP REST / Webhook endpoint.

Supporting these endpoints opens many possibilities to integrate directly with external platforms, in real-time.

[svelde]

As a source, offer both consuming a 3rd party endpoint (where the Eventstream pulls data from) and providing an endpoint where 3rd party solutions push data to. That last variant should act like an HTTP-triggered function.

[alexlzx]

Thanks for your suggestion to add HTTP REST / Webhook support to Eventstream. It’s a valuable idea for enabling real-time integration with external platforms.

To confirm, are you looking for the following capabilities?

1. HTTP source (pull-based): Eventstream continuously pulls data from external HTTP endpoints.
2. HTTP source (push-based): Eventstream acts as a webhook receiver, allowing external platforms to send HTTP POSTs or other requests to it.
3. HTTP destination: Eventstream sends data to external systems via HTTP.

Let us know if this matches your request, or if there are specific platforms or use cases you’d like us to consider.

[svelde]

Yes, all three are useful. Regarding pulling data from a source (eg. time based) is perfect. This can lead to duplicates but this can be fixed using eg. Eventhouse materialized view. Allowing some control over the query properties would be helpful eg. the datetime of the LAST successful call. I can think of more elaborate scenarios (like a datetime from the last row from the previous message body indicating the latest messages received at that moment so no messages are missed).

[svelde]

See also this Fabric community discussion: https://community.fabric.microsoft.com/t5/Eventstream/Options-for-High-Frequency-Low-Latency-API-Ing... There dynamic behaviour is expected so the latest call starts where the previous one ended via a (timestamp) parameter. This way, gaps in ingestion or overlap or an ingestion overload (when a burst of data is provided) can be prevented.