Dataflow gen 2 Incremental Refresh(Bucket Size and Concluded Periods)

Hi fabconmvp 

For your first question, If you set a bucket size (e.g., Month or Year), the dataflow will create these “buckets” for periods of data based on the size you specified. When data refreshes, only buckets with data modified since the last refresh will be processed. For example:
With a Year bucket size, the data from 2022 to 2024 will be divided by year, and only the latest year's bucket will be refreshed if only new data is added for that year.

For your second question, the "Only extract data for concluded periods" option affects what data is processed during each refresh:
When Enabled: The dataflow will only refresh data from completed periods based on your bucket size. For instance, if your bucket size is a month, and today is November 6, 2024, the refresh will process only data from months that have fully concluded (up to October 2024). It won’t include partial data from November 2024, as the month is still ongoing.
When Disabled: The dataflow can include data from periods that are not fully concluded (e.g., partial data for the current month). However, it may not fetch all data if certain date sequences are not incrementing as expected (e.g., gaps in data or non-sequential dates within the bucket size). If, for instance, there’s an inconsistency in the DateTime sequence within your specified range, the dataflow may only fetch the most recent continuous dates up to the last concluded date (e.g., 01/01/2020 - 11/30/2024 in your case).

Best Regards,

Jayleny

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.