Forum Discussion
Incremental Data Load Options in Microsoft Fabric
- Anonymous2 years ago
Hi jashfabric
Thanks for using Fabric Community.
You can use a Dataflow gen2 for incremental load. Incrementally amassing data in a data destination requires a technique to load only new or updated data into your data destination. This technique can be done by using a query to filter the data based on the data destination.
For more information please refer to this link:
https://learn.microsoft.com/en-us/fabric/data-factory/tutorial-setup-incremental-refresh-with-dataflows-gen2
Hope this helps. Please let me know if you have any further queries. - 2 years ago
Hi jashfabric from a Spark/Notebook/Delta perspective, you could look into using Change Data Feed:
Delta Lake Change Data Feed in Fabric Lakehouses (serverlesssql.com)
Hi jashfabric from a Spark/Notebook/Delta perspective, you could look into using Change Data Feed:
Delta Lake Change Data Feed in Fabric Lakehouses (serverlesssql.com)