Incremental refresh - desktop vs. dataflow

Hi,

I'm facing the same issue with dataflow incremental refresh. When enabling incremental refresh on one of my dataflow tables, it auto-generates RangeStart and RangeEnd "parameters" (which are not seen as parameters in the manage parameters dialog) crashing the editor if I already had explicit RangeStart and RangeEnd parameters in my query, and adds a filter step at the end of the query's steps to filter the selected datetime field >= RangeStart and < RangeEnd.

It lets you modify the steps and reorder things, but when you save and come back, the last auto-generated filter is back (if you had deleted it or moved it).

I tried however to work with it, but to no avail.

I'm sourcing data from a Web.Contents connector with startdate and enddate parameters ; I'm using RangeStart and RangeEnd values to feed the Web.Contents' startdate and enddate.

I'm keeping the last step auto-generated filter ; it won't filter anything as it will be filtered at the Web.Contents step but I don't mind keeping it if the dataflow is happier with it.

The preview is working fine (the auto-generated RangeStart and RangeEnd have default values of 01/01/0001 and 31/12/9999), the Web.Contents is returning rows between these two extreme dates, and I can validate the dataflow.

However, when I try to refresh the dataflow, I get an error.

Looking at the error, it seems to me that the Web.Contents source didn't return any row, as if empty parameters were provided to startdate and enddate (through auto-generated RangeStart and RangeEnd).

Questions :

Are the auto-generated RangeStart and RangeEnd in dataflow incremental refresh only accessible to the auto-generated filter step or can they be referenced in other steps, like mine on Web.Contents source?

Is there a way to check what values they were given so I can understand why the refresh would fail? (I setup the incremental refresh with 1 year data retention and 30 days refresh period, and I have data available on both 1 year and the last 30 days, so the Web.Contents step shouldn't return empty results neither for the initial load nor for the incremental ones, hence my suspicion of empty/unreferencable RangeStart/RangeEnd auto-generated parameters)

Has someone been successful setting up dataflow incremental refresh with a Web.Contents source?

Regards,

Florent