Forum Discussion

raghulvl's avatar
raghulvl
Frequent Visitor
3 months ago
Solved

SQL Server CDC on VM DB Source Fails with "Property 'username' value '' is invalid" in Fabric Events

Hi Community, I am trying to build a real-time ingestion pipeline in Microsoft Fabric using the following architecture: SQL Server (On-Premises) in my local laptop → Eventstream (SQL Server CDC on ...
Real-Time hub
  • raghulvl's avatar
    raghulvl
    3 months ago

    Hey v-dineshya ,

    I wanted to share an update that I have successfully completed my Proof of Concept (POC) for streaming real-time data from SQL Server into Microsoft Fabric.

    After evaluating a few approaches, I achieved the desired low-latency pipeline by implementing the following architecture:

    SQL CDC → (Debezium + Kafka) → (Azure Event Hubs / Confluent.io) → Fabric Eventstream (via Event Hub or Confluent Connector)

    Why this approach worked well:

    • Log-Based CDC: By using Debezium, the pipeline reads from the SQL transaction logs directly, ensuring zero impact on primary database query performance.

    • Scalability: Routing the data through Kafka/Event Hubs provides a highly reliable, buffered streaming layer.

    • Native Integration: Fabric’s native Eventstream connectors made it incredibly straightforward to ingest the stream directly into the lakehouse for real-time analytics.

    Thanks to everyone for the guidance and support throughout this process! Hopefully, this architectural flow helps anyone else looking to implement a similar real-time CDC pattern.

    Cheers!

raghulvl's avatar
raghulvl
Frequent Visitor
3 months ago

Hi deborshi_nag,

Thank you for taking the time to respond to my query.

I reviewed the documentation you shared and noticed that the current connector support is limited to Azure VMs and similar cloud-hosted virtual machines.

As an alternative, I am currently exploring the following approach:

SQL Server CDC → Debezium → Kafka (Confluent) → Event Stream

I'm hoping this setup will work for my use case. Since I am learning and experimenting with the technology, I am also trying to leverage free trial options wherever possible.

Thanks again for your guidance. I will also try the public VM approach you suggested and will keep you posted on the results.

deborshi_nag's avatar
deborshi_nag
Icon for Super User rankSuper User
3 months ago

Your alternative architecture should work - you're running Debezium close to yuor laptop and using Kafka as a bridge to the cloud - let us know how it goes - good pattern when souce is not publicly accessible to cloud!