Forum Discussion
CDC implementation for Dataverse table in fabric using delta links
Hi Ymatole, dataverse tables do include a watermark column that is the modifiedon field, which records the last update timestamp for each row. This makes incremental loading straightforward by filtering rows where modifiedon is greater than the last processed timestamp. For a more robust CDC approach (including deletes), you can enable Change Tracking in Dataverse.
But, an even more efficient option is to use Fabric Link for Dataverse, which streams changes into Fabric Lakehouse in near real time, eliminating the need for manual incremental logic. You can learn more about this approach here: https://learn.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-view-in-fabric.Hope this helps. If so, please give kudos 👍 and mark as Accepted Solution ✔️ to help others.
Hi Ymatole,
Thank you nielsvdc for your response.
Along with the previous points, remember that when using incremental loads based on modifiedon, it’s important to store the last successfully processed timestamp externally, such as in a control table or pipeline parameter, to prevent data gaps or duplicates during retries.
If you need high-volume or near–real-time sync, be sure to check Dataverse API limits and throttling, as these can affect large incremental loads. Fabric Link can assist, but monitoring sync latency and full-load performance is still necessary.
Finally, make sure that Change Tracking is enabled at the table level and is supported for your specific Dataverse entities, since not all system or virtual tables support CDC in the same way.
Thank you.
Hi Ymatole, dataverse tables do include a watermark column that is the modifiedon field, which records the last update timestamp for each row. This makes incremental loading straightforward by filtering rows where modifiedon is greater than the last processed timestamp. For a more robust CDC approach (including deletes), you can enable Change Tracking in Dataverse.
But, an even more efficient option is to use Fabric Link for Dataverse, which streams changes into Fabric Lakehouse in near real time, eliminating the need for manual incremental logic. You can learn more about this approach here: https://learn.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-view-in-fabric.
Hope this helps. If so, please give kudos 👍 and mark as Accepted Solution ✔️ to help others.
So i tried this streaming option and facing concerns with 2 days of default retention of checkpoint in fabric. If we don't get enough updates on the table within 2 days, the checkpoint gets deleted automatically and the process fails. Any suggestions on this ?
Hi Ymatole,
This behavior is normal with the current Fabric Link streaming model. Fabric keeps an internal checkpoint for about 48 hours, and if no new changes come from Dataverse during this time, the checkpoint is deleted. When this happens, the stream can't resume and the process fails. Currently, there's no way to extend this retention period in Fabric. For tables with few or infrequent updates, streaming may not be ideal. To address this, you can either keep the stream active by making at least one change within the retention window, or use a fallback incremental load based on the modifiedon column or Change Tracking, saving the last processed watermark externally. Many teams choose to use only incremental loads for such tables, as this method is simpler and more reliable for low-volume entities.
Thank you.
