I have a dataflow that I am trying to increase performance with now that our dataset is getting larger. I am testing the dataflow incremental refresh capabilities by trying it on one table using a ModifiedTS column and pulling the last 5 years worth of data but only updating data within the past month. When I do a full refresh, the refresh of this one table takes 3.5 minutes and 72000 rows are pulled in. When I configure incremental refresh per Adam Saxton's dataflow native query method, the query works, partitions are made, but the first refresh does not pull in all of the data. Only 1205 rows are brought in and the data is from the beginning of the 5 years and not even the last month. When looking at the refresh details export, I can see the partitions and it reflects the 609 bytes in 2018, but "N/A" for all other partitions. I also noticed that the RangeStart and RangeEnd in Power Query show as "01/01/01 00:00am" and "12/31/9999 23:59pm".
Any ideas on what might be causing these issues. Any assistance is appreciated.
