Future date are missing in the dataset when we are using the incremental refresh policy.

Hi Anonymous ,

According to your configurations, when you refresh the dataset, firstly , will check if the SQDRRefreshTime has changed, if yes, will according to the current time to refresh the 18 months before the refresh date.

The Archive data starting before refresh date setting determines the historical period in which rows with a date/time in that period are included in the dataset, plus rows for the current incomplete historical period, plus rows in the refresh period up to the current date and time.

For example, if we specify 5 years, our table will store the last five whole years of historical data in year partitions, plus rows for the current year in quarter, month, or day partitions, up to and including the refresh period.

The Incrementally refresh data starting before refresh date setting determines the incremental refresh period in which all rows with a date/time in that period are included in the refresh partition(s) and refreshed with each refresh operation.

For example, if we specify a refresh period of 3 days, with each refresh operation the service overrides the RangeStart and RangeEnd parameters to create a query for rows with a date/time within a three-day period, beginning and ending dependent on the current date and time. Rows with a date/time in the last 3 days up to the current refresh operation time are refreshed. With this type of policy, we can expect our FactInternetSales dataset table in the service, which averages 10k new rows per day, to refresh roughly 30k rows with each refresh operation.

Best regards,
Community Support Team_ Binbin Yu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.