Forum Discussion
Issues with Azure PostgreSQL CDC to Eventhouse (KQL) using EventStream
Hello Everyone,
We are streaming data from Azure PostgreSQL CDC to Eventhouse and encountering issues with missing or duplicate records, likely due to inconsistencies in how CDC events are captured, processed, or committed into Fabric OneLake.
The input from PostgreSQL into Event Stream appears correct, but when the data is stored in KQL, we are seeing either fewer or more records than expected.
Has anyone used Azure PostgreSQL as a source connector with KQL as the destination? We are unable to use Lakehouse as the destination due to a conversion error, which the Microsoft PG team is currently investigating.
Any insights would be greatly appreciated.
Thank you!
Hi PrachiJain_2025 ,
I tested it myself according to the steps in this document, but I didn't find any problems.
Add PostgreSQL Database CDC source to an eventstream - Microsoft Fabric | Microsoft Learn
However, the amount of data I tested was small. How big is your data? Are there multiple data tables involved?
You can try to modify some restriction policies in the KQL database using these commands and then re-import to see if the problem still exists.
Request limits policy - Kusto | Microsoft Learn
Also, I saw you mentioned:the Microsoft PG team is currently investigatingIf you find a solution in your feedback, please remember to reply the problem and solution below so that other users can find the solution faster. Thank you!
Best Regards,
Dino Tao
1 Reply
Hi PrachiJain_2025 ,
I tested it myself according to the steps in this document, but I didn't find any problems.
Add PostgreSQL Database CDC source to an eventstream - Microsoft Fabric | Microsoft Learn
However, the amount of data I tested was small. How big is your data? Are there multiple data tables involved?
You can try to modify some restriction policies in the KQL database using these commands and then re-import to see if the problem still exists.
Request limits policy - Kusto | Microsoft Learn
Also, I saw you mentioned:the Microsoft PG team is currently investigatingIf you find a solution in your feedback, please remember to reply the problem and solution below so that other users can find the solution faster. Thank you!
Best Regards,
Dino Tao