topic Dataflow gen2 - Incremental refresh in Dataflow

Dataflow gen2 - Incremental refresh

msprog — Thu, 28 Aug 2025 12:19:04 GMT

Hello,

I am trying Dataflow Gen 2 with incremental refresh.

I read a SQL table via gateway and populate a lakehouse table. There are no transofrmations.

If i run the dataflow without any incremental refresh configured , it runs fine.

However, if i enable incremental refresh, it fails. the error is below.

the query script is simple:

let

Source = Sql.Database("servername", "AdventureWorksDW2019"),

#"Navigation 1" = Source{[Schema = "dbo", Item = "FactInternetSalesBkp"]}[Data]

#"Navigation 1"

error is:

There was a problem refreshing the dataflow: 'Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: The value does not support versioning. Details: Reason = Expression.Error;ErrorCode = 10855;Detail = #table({"ProductKey", "OrderDateKey", "DueDateKey", "ShipDateKey", "CustomerKey", "PromotionKey", "CurrencyKey", "SalesTerritoryKey", "SalesOrderNumber", "SalesOrderLineNumber", "RevisionNumber", "OrderQuantity", "UnitPrice", "ExtendedAmount", "UnitPriceDiscountPct", "DiscountAmount", "ProductStandardCost", "TotalProductCost", "SalesAmount", "TaxAmt", "Freight", "CarrierTrackingNumber", "CustomerPONumber", "OrderDate", "DueDate", "ShipDate"}, {});Microsoft.Data.Mashup.Error.Context = User GatewayObjectId: a0715ee3-xxxx-4a77-912b-3bf5ebc598c0'. Error code: Mashup Exception Expression Error. (Request ID: a98070c6-6d8e-4961-88f0-xxxxxx).

Please does anyone has experience with this.

Re: Dataflow gen2 - Incremental refresh

Mauro89 — Thu, 28 Aug 2025 14:46:51 GMT

Hi msprog,

Incremental refresh isn’t supported when the Dataflow Gen2 output goes straight to a Lakehouse. Right now, the supported destinations for incremental refresh are:

Fabric Warehouse
Fabric SQL endpoint
Azure SQL Database

So if you need incremental refresh, you’ll need to land the data in one of those targets instead of directly in the Lakehouse.

Workarounds:

Keep loading into the Lakehouse, but implement your own “delta logic” in Power Query or pipelines (e.g. filter on modified date and append only new rows).
Or, load into a Warehouse with incremental refresh enabled, and if needed, create a shortcut from that Warehouse back to the Lakehouse.

If this helps, consider marking the post as solution.

Best regards!

Re: Dataflow gen2 - Incremental refresh

msprog — Mon, 01 Sep 2025 00:45:41 GMT

Hey thanks for this. But the document says its in preview? https://learn.microsoft.com/en-us/fabric/data-factory/dataflow-gen2-incremental-refresh

Re: Dataflow gen2 - Incremental refresh

Anonymous — Mon, 01 Sep 2025 05:38:00 GMT

Hi msprog ,

Documentation lists Fabric Lakehouse (preview) as supported for incremental refresh but it’s not fully functional yet.

For now, incremental refresh works reliably only with other GA destinations.

Keep an eye on Microsoft Fabric Roadmap Microsoft’s roadmap as Lakehouse incremental refresh continues to evolve toward production readiness.

Regards,
Akhil.

Re: Dataflow gen2 - Incremental refresh

Anonymous — Thu, 04 Sep 2025 09:00:20 GMT

Hi msprog ,

Did you get a chance to review the note about Lakehouse incremental refresh being in preview? For now, GA destinations are the safe bet. Curious if you’ve considered one of those as a temporary workaround until Lakehouse support matures.

Regards,
Akhil.