Forum Discussion
Solved
Dataflow gen2 - Incremental refresh
Hello, I am trying Dataflow Gen 2 with incremental refresh. I read a SQL table via gateway and populate a lakehouse table. There are no transofrmations. If i run the dataflow without any incr...
Hi msprog,
Incremental refresh isn’t supported when the Dataflow Gen2 output goes straight to a Lakehouse. Right now, the supported destinations for incremental refresh are:
- Fabric Warehouse
- Fabric SQL endpoint
- Azure SQL Database
So if you need incremental refresh, you’ll need to land the data in one of those targets instead of directly in the Lakehouse.
Workarounds:
- Keep loading into the Lakehouse, but implement your own “delta logic” in Power Query or pipelines (e.g. filter on modified date and append only new rows).
- Or, load into a Warehouse with incremental refresh enabled, and if needed, create a shortcut from that Warehouse back to the Lakehouse.
If this helps, consider marking the post as solution.
Best regards!
Hi msprog ,
Documentation lists Fabric Lakehouse (preview) as supported for incremental refresh but it’s not fully functional yet.
For now, incremental refresh works reliably only with other GA destinations.
Keep an eye on Microsoft Fabric Roadmap Microsoft’s roadmap as Lakehouse incremental refresh continues to evolve toward production readiness.
Regards,
Akhil.