Confusing Documentation For DataFlow Gen2 Incremental Refresh

Hi kgardner3300 

Your high-level steps are mostly correct, but here are a few clarifications to ensure there’s no confusion:

Step 1: Detecting Changes via ModifiedDate

• When you check “Only extract new data when the maximum value in this column changes” and point it at ModifiedDate, Dataflow Gen2 will query the full source table to see if any row’s ModifiedDate is greater than the last time it ran.

• If nothing has changed, the incremental refresh stops immediately, and no partitions are touched.

• If at least one row’s ModifiedDate has increased, Fabric proceeds to re-evaluate the defined partition range based on OrderDate.

Step 2: Partitioning by OrderDate

• Once a change is detected, Fabric builds partitions (buckets) by splitting the OrderDate column into daily slices (because you chose “2 Weeks” / “Bucket size: Day”).

• In your example, that creates 14 separate partitions—one for each day in the last two weeks (Day – 0 through Day – 13).

• Each partition is processed in parallel, so Fabric can read and transform just those 14 days instead of scanning the entire table.

Step 3: Replacing Only the 2-Week Window in the Destination

• For each of those 14 daily partitions, Fabric issues a “write” (upsert/replace) against the destination entity, deleting and re-inserting all rows whose OrderDate falls within that one-day slice.

• Rows whose OrderDate is older than two weeks are left untouched—Fabric never rewrites those partitions.

• In other words, only rows in the “past two weeks” window (as of today) are re-loaded; everything older remains as‐is in the destination.

If the above information helps you, please give us a Kudos and marked the Accept as a solution.

Best Regards,
Community Support Team _ C Srikanth.