Forum Discussion
CDC implementation for Dataverse table in fabric using delta links
I am looking for efficient incremental load method for dataverse tables. Not sure if dataverse tables have watermark column or any other efficient way available to implement CDC.
Hi Ymatole, dataverse tables do include a watermark column that is the modifiedon field, which records the last update timestamp for each row. This makes incremental loading straightforward by filtering rows where modifiedon is greater than the last processed timestamp. For a more robust CDC approach (including deletes), you can enable Change Tracking in Dataverse.
But, an even more efficient option is to use Fabric Link for Dataverse, which streams changes into Fabric Lakehouse in near real time, eliminating the need for manual incremental logic. You can learn more about this approach here: https://learn.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-view-in-fabric.Hope this helps. If so, please give kudos 👍 and mark as Accepted Solution ✔️ to help others.
Hi Ymatole,
Thank you nielsvdc for your response.
Along with the previous points, remember that when using incremental loads based on modifiedon, it’s important to store the last successfully processed timestamp externally, such as in a control table or pipeline parameter, to prevent data gaps or duplicates during retries.
If you need high-volume or near–real-time sync, be sure to check Dataverse API limits and throttling, as these can affect large incremental loads. Fabric Link can assist, but monitoring sync latency and full-load performance is still necessary.
Finally, make sure that Change Tracking is enabled at the table level and is supported for your specific Dataverse entities, since not all system or virtual tables support CDC in the same way.
Thank you.
7 Replies
Hi Ymatole, dataverse tables do include a watermark column that is the modifiedon field, which records the last update timestamp for each row. This makes incremental loading straightforward by filtering rows where modifiedon is greater than the last processed timestamp. For a more robust CDC approach (including deletes), you can enable Change Tracking in Dataverse.
But, an even more efficient option is to use Fabric Link for Dataverse, which streams changes into Fabric Lakehouse in near real time, eliminating the need for manual incremental logic. You can learn more about this approach here: https://learn.microsoft.com/en-us/power-apps/maker/data-platform/azure-synapse-link-view-in-fabric.Hope this helps. If so, please give kudos 👍 and mark as Accepted Solution ✔️ to help others.
So i tried this streaming option and facing concerns with 2 days of default retention of checkpoint in fabric. If we don't get enough updates on the table within 2 days, the checkpoint gets deleted automatically and the process fails. Any suggestions on this ?
Hi Ymatole,
This behavior is normal with the current Fabric Link streaming model. Fabric keeps an internal checkpoint for about 48 hours, and if no new changes come from Dataverse during this time, the checkpoint is deleted. When this happens, the stream can't resume and the process fails. Currently, there's no way to extend this retention period in Fabric. For tables with few or infrequent updates, streaming may not be ideal. To address this, you can either keep the stream active by making at least one change within the retention window, or use a fallback incremental load based on the modifiedon column or Change Tracking, saving the last processed watermark externally. Many teams choose to use only incremental loads for such tables, as this method is simpler and more reliable for low-volume entities.
Thank you.
Hi Ymatole,
Thank you nielsvdc for your response.
Along with the previous points, remember that when using incremental loads based on modifiedon, it’s important to store the last successfully processed timestamp externally, such as in a control table or pipeline parameter, to prevent data gaps or duplicates during retries.
If you need high-volume or near–real-time sync, be sure to check Dataverse API limits and throttling, as these can affect large incremental loads. Fabric Link can assist, but monitoring sync latency and full-load performance is still necessary.
Finally, make sure that Change Tracking is enabled at the table level and is supported for your specific Dataverse entities, since not all system or virtual tables support CDC in the same way.
Thank you.
Hi Ymatole,
we haven't heard back from you regarding our last response and wanted to check if your issue has been resolved.Should you have any further questions, feel free to reach out.
Thank you for being a part of the Microsoft Fabric Community Forum!Hi Ymatole,
I wanted to follow up on our previous suggestions regarding the issue. We would love to hear back from you to ensure we can assist you further.
Thank you.
Hi Ymatole
You can achieve CDC with Dataverse. The key concept is zero‑copy integration via OneLake shortcuts created by the Link to Microsoft Fabric feature—this exposes Dataverse tables in a Fabric lakehouse as Delta/Parquet without exporting or ETL, and keeps them continuously in sync.Your options are:Link to Microsoft Fabric
- From Power Apps, you link a Dataverse environment to a Fabric workspace. Fabric automatically creates a Lakehouse, SQL endpoint, Power BI dataset, and OneLake shortcuts for your Dataverse tables. Changes in Dataverse flow through and appear in Fabric with no copies or pipelines.
Create Dataverse shortcuts directly from Fabric
- If you want to pick specific tables (instead of bringing the whole environment), you can create shortcuts to Dataverse inside a Lakehouse directly in Fabric. This still uses the Dataverse managed lake behind the scenes and is read‑only
Existing Azure Synapse Link → Fabric
- If you already use Synapse Link for Dataverse to land Delta tables in ADLS Gen2, you can create a OneLake shortcut to that ADLS from your Fabric Lakehouse, reusing your existing link profile—so you can adopt Fabric without changing your current extraction path.
Hope this gives you an overview on options available on Fabric for you. If this helps, please Accept as a solution.
