Forum Discussion
CDC implementation for Dataverse table in fabric using delta links
- 7 months ago
Hi Ymatole, dataverse tables do include a watermark column that is the modifiedon field, which records the last update timestamp for each row. This makes incremental loading straightforward by filtering rows where modifiedon is greater than the last processed timestamp. For a more robust CDC approach (including deletes), you can enable Change Tracking in Dataverse.
But, an even more efficient option is to use Fabric Link for Dataverse, which streams changes into Fabric Lakehouse in near real time, eliminating the need for manual incremental logic. You can learn more about this approach here: https://learn.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-view-in-fabric.Hope this helps. If so, please give kudos 👍 and mark as Accepted Solution ✔️ to help others.
- 7 months ago
Hi Ymatole,
Thank you nielsvdc for your response.
Along with the previous points, remember that when using incremental loads based on modifiedon, it’s important to store the last successfully processed timestamp externally, such as in a control table or pipeline parameter, to prevent data gaps or duplicates during retries.
If you need high-volume or near–real-time sync, be sure to check Dataverse API limits and throttling, as these can affect large incremental loads. Fabric Link can assist, but monitoring sync latency and full-load performance is still necessary.
Finally, make sure that Change Tracking is enabled at the table level and is supported for your specific Dataverse entities, since not all system or virtual tables support CDC in the same way.
Thank you.
Hi Ymatole,
Thank you nielsvdc for your response.
Along with the previous points, remember that when using incremental loads based on modifiedon, it’s important to store the last successfully processed timestamp externally, such as in a control table or pipeline parameter, to prevent data gaps or duplicates during retries.
If you need high-volume or near–real-time sync, be sure to check Dataverse API limits and throttling, as these can affect large incremental loads. Fabric Link can assist, but monitoring sync latency and full-load performance is still necessary.
Finally, make sure that Change Tracking is enabled at the table level and is supported for your specific Dataverse entities, since not all system or virtual tables support CDC in the same way.
Thank you.
Hi Ymatole,
we haven't heard back from you regarding our last response and wanted to check if your issue has been resolved.
Should you have any further questions, feel free to reach out.
Thank you for being a part of the Microsoft Fabric Community Forum!
- v-sgandrathi7 months agoCommunity Support
Hi Ymatole,
I wanted to follow up on our previous suggestions regarding the issue. We would love to hear back from you to ensure we can assist you further.
Thank you.