Forum Discussion

craigmday's avatar
craigmday
Icon for Helper I rankHelper I
1 year ago

Dataflow Gen 2: Incremental Refresh fails with error: The value does not support versioning

Hi all,   I am trying to configure incremental refresh on a data source that contains financial entries (general ledger entries). I am using a Dataflow Gen 2 and am publishing to a Fabric Lakehouse...
data warehouse
warehouse
tanveerSheikh's avatar
tanveerSheikh
New Member
1 year ago

Hi craigmday,

 

That error usually comes up because of how Dataflow Gen2 handles versioning when writing to a Lakehouse/Warehouse. With incremental refresh, Fabric tries to write only deltas, but the destination table isn’t supporting the versioning process. That’s why it works with full refresh but fails with incremental.

 

A couple of things you can check/try:

  • Make sure you have a stable unique key (for example, TransactionID + TransactionDate) so Fabric knows how to identify rows.
  • Keep your Dataflow very lean – instead of applying incremental refresh there, just load the latest slice of data into your Lakehouse Bronze layer.
  • Then, use the Medallion pattern:
    • Bronze = raw latest data
    • Silver = transformations/clean-up
    • Gold = full historical view

This way, Dataflow is only handling small, fast loads, and your Lakehouse manages history. It usually performs much better and avoids this type of error.

 

Hope this helps — let us know how it goes 👍

craigmday's avatar
craigmday
Icon for Helper I rankHelper I
1 year ago

Hi Tanveer,

Thanks very much for the feedback. The data source has a compound primary key consisting of six different fields. I tried including all fields to ensure the full key value was present, but got the same error. I tried adding a merge column which combined the six columns into one, however, this part of the query does not fold, so I can't publish with incremental refresh.

 

Do you know if I need to have a single unique key column in my data source for this work?

 

Thanks. 

  • Anonymous's avatar
    Anonymous
    Not applicable
    1 year ago

    Hi craigmday ,

    Please consider reaching out to Microsoft Support. You can provide them with all the troubleshooting steps you've already taken, which will help them understand the issue better and provide a resolution. They might be able to identify something specific about your admin account setup or provide a solution that isn't immediately obvious. 

    Below is the link to create Microsoft Support ticket:
    How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn

    Thank you