Enable a way to set incremental refresh in Dataflows

[#1]

Enable an experience that allows the end-user to setup incremental refresh for Dataflows Gen2 in Data Factory for Microsoft Fabric

[fbcideas_migusr]

Add a Table Action option for Incremental Load.

• Let the user identify the Primary Key column(s) from the source (or read meta data from source)
• Add some auditing columns (is current, is active, is deleted, load date, update date, current key)
• Add new rows, update existing rows, mark deleted rows.

Or is some of this plumbing already taken care of by the delta tables?

[fbcideas_migusr]

Incremental refreshes are sorely needed when getting data from on-prem SQL servers. Not having this ability can be (is) a deal breaker.

[shameer]

Fabric is in GA, and we are eagerly awaiting IR for Dataflows Gen2. As we're on the verge of 2023, do we have any updates on the ETA?

[cko2]

Any update on this? Is this still planned for second half of 2023?

An extremely essential function

[Rozzano_Marco]

We're eager to use incremental refresh in dataflows gen 2. Any hint about when we can expect this critical feature ? The second half of 2023 is almost over...

[fbcideas_migusr]

Hi. I agree, a support for the incremental refresh is absolutely a "must have"!

In addition to that, let users:

1. to identify Key column(s) from the source (or read meta data from source)
2. to identify date(time) columns such as: CreatedOn, ModifiedOn, LastModifiedOn
3. to choose where to drop the result in the Tables area of Files area

BR,

Predrag

[Stefan_Berreite]

Hi Microsoft Fabric Team,

just checking in as it is vital to our projects. Can we expect incremental refresh in Dataflows Gen2 to be GA in Q2 (or will the release be pushed back again)?

BR, Stefan

[David_COURTENS]

Absolute must-have feature in order to use the Gen2 Dataflows.

Also in my opinion surprising to bring a Gen2 product that is not 'down-worth compatible' with a Gen1 product.

Hopefully Q2.2024 is still the feasible deadline.

kr,

David

[jsmith2]

Any update on this? It's now after Q2 2024

[Bryan_Rock]

Unfortunately, the Data Factory roadmap now says Q3 2024.

https://learn.microsoft.com/en-us/fabric/release-plan/data-factory