Eventstream from On-Premise (Azure VM) SQL Server CDC?

[rockfish]

We have SQL Server Enterprise running on an Azure VM.  While we are able to create a gateway, there does not appear to be a connector to bring it through an Evenstream.  Our objective is to keep our (logical) bronze layer in sync with the source database.   CDC seemed like the best approach.

- Is it possible to ingest CDC data from on-prem SQL Server using Eventstream?

- If not, can anyone recommend the next best alternative?

Thanks!

Jeff

[sblake-stjwa]

Microsoft solution: Add SQL Server Change Data Capture as a source to eventstream - Microsoft Fabric | Microsoft Learn

Non-Microsoft but open source options, both require CDC turned on on the SQL Server:

- Kafke with Kafke Connect with a Debezium connector to retrieve the changing data from the SQL Server as a Source, then the Apache Kafka endpoint from Event Streaming as Sink.

- Apache Pulsar with the Debezium connector to retrieve the changing data from the SQL Server as a Source, then the Apache Kafka endpoint from Event Streaming as Sink.

Add Apache Kafka source to a Fabric eventstream - Microsoft Fabric | Microsoft Learn

[xujx]

@rockfish Ingesting cdc data from on-prem SQL Server with Eventstream is in our roadmap of Eventstream. We are actively researching on this scenario to work out the best experience for this scenario. 

When you say "Our objective is to keep our (logical) bronze layer in sync with the source database", how do you bring your initial snapshot to the target bronze layer from the source database? using certain batch copy job or you expect the cdc to bring the initial snapshot to bronze layer as well?

[Anonymous]

Hi @rockfish ,

Currently, Eventstream primarily supports data ingestion from sources like Azure Event Hubs, Azure IoT Hub, sample data, and custom apps. Direct ingestion of CDC data from an on-premise SQL Server into Eventstream is not supported out-of-the-box.
For more details on supported sources by Eventstream, you can refer to the documentation here:
https://learn.microsoft.com/en-us/fabric/real-time-intelligence/event-streams/add-manage-eventstream... 

If you must use EventStream, please use your data in a data source supported by EventStream.

Best Regards,
Dino Tao
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.