Forum Discussion
Incremental Data Load Options in Microsoft Fabric
Hi jashfabric
Thanks for using Fabric Community.
You can use a Dataflow gen2 for incremental load. Incrementally amassing data in a data destination requires a technique to load only new or updated data into your data destination. This technique can be done by using a query to filter the data based on the data destination.
For more information please refer to this link:
https://learn.microsoft.com/en-us/fabric/data-factory/tutorial-setup-incremental-refresh-with-dataflows-gen2
Hope this helps. Please let me know if you have any further queries.Hi jashfabric from a Spark/Notebook/Delta perspective, you could look into using Change Data Feed:
Delta Lake Change Data Feed in Fabric Lakehouses (serverlesssql.com)
Hi jashfabric
Thanks for using Fabric Community.
You can use a Dataflow gen2 for incremental load. Incrementally amassing data in a data destination requires a technique to load only new or updated data into your data destination. This technique can be done by using a query to filter the data based on the data destination.
For more information please refer to this link:
https://learn.microsoft.com/en-us/fabric/data-factory/tutorial-setup-incremental-refresh-with-dataflows-gen2
Hope this helps. Please let me know if you have any further queries.
Thanks for your help Anonymous AndyDDC
That really helps.
Hi jashfabric
Glad that your query got resolved. Please continue using Fabric Community for any help regarding your queries.