SQL CDC to Fabric

[J-Schreck]

Scenario:

On-prem SQL db with CDC logging enabled 
Fabric is all setup 

Connections to the SQL db is all good but when I try to use the LSN values __$Start_LSN for referance for the CDC ingestion, they are in binary format and cant be read by any tool I have tried in fabric. 

It dont need real time, just CDC. 

[v-sgandrathi · Accepted solution]

Hello @J-Schreck,

The challenge with __$Start_LSN values appearing in binary format is expected behavior within SQL Server CDC. Since Fabric doesn’t handle this format directly, a practical approach is to convert the binary LSN values into a readable format (such as hexadecimal or timestamps) using SQL Server’s built-in functions, like CONVERT() or sys.fn_cdc_map_lsn_to_time()
sys.fn_cdc_map_lsn_to_time (Transact-SQL) - SQL Server | Microsoft Learn.

For periodic CDC ingestion — given that real-time processing isn’t a requirement — Dataflows Gen2 or Data Factory pipelines are effective solutions within Fabric. These methods support scheduled data pulls and transformations.

To ensure you capture only new changes, I recommend tracking the last processed LSN in a Fabric table and using it as a reference point to fetch only incremental data on each load.

This approach should resolve the binary LSN issue and ensure smooth, repeatable CDC ingestion into Fabric.

I hope this could resolve you issue, if you need any further assistance, feel free to reach out.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you for using Microsoft Community Forum.

[v-sgandrathi]

Hi @J-Schreck,

I wanted to follow up on our previous suggestions regarding the issue. We would love to hear back from you to ensure we can assist you further.

If our response has addressed your query, please Accept it as a solution and give a ‘Kudos’ so other members can easily find it. Please let us know if there’s anything else we can do to help.

Thank you.

[v-sgandrathi]

Hi @J-Schreck,

May I ask if you have gotten this issue resolved?

If it is solved, please mark the helpful reply or share your solution and accept it as solution, it will be helpful for other members of the community who have similar problems as yours to solve it faster.

Thank you.

[v-sgandrathi]

Hi @J-Schreck,

we haven't heard back from you regarding our last response and wanted to check if your issue has been resolved. If our response addressed by the community member for  your query, please mark it as Accept Answer and click Yes if you found it helpful.

Should you have any further questions, feel free to reach out.

Thank you for being a part of the Microsoft Fabric Community Forum!

[ajarora]

We are building native CDC based ingestion from various data stores like SqlOnPrem through CopyJob.

This capability is in private preview right now, and will soon go live. Please DM me for more details.

[Jeff_Schreck]

I have not found a good solution at this time. 

I have 3 significant data sources I am trying to compelte this operation with. I dont have direct access to the SQL servers to change the datatypes at source. 

Jeff