Forum Discussion
Power BI incremental refresh problem
hi jtclabaugh
The reason it runs the top 1000 is because it first validates that the data that is going to be returned matches the table definition.
Otherwise it would have to run the entire query only to fail at the end.
This top 1000 query should be quick, and it will only be run once for each partition created.
Hmmm. This is not going to work. #1, it's running more than one preview query per parition when running the incremental (looks like one for each power query step whether these are folded depends on the step). #2, this is only happening with incremental, for loads without incremental turned on, it issues no preview queries. #3, it does not do this with incremental for dataflows. #4, these are not fast, the datasource has 100's of millions of records - folded queries run fast, select * w/ top 1000 do not. #5, I need to go back and check 100+history partitions each time I run the load as historical data can change. Going to try Table.View with OnTake argument, but not sure if this will work running from the service. Can't imagine this is the way incremental was designed to work.
Hey jtclabaugh,
Did you find out more about this issue?
Also, how did your try with Table.view with Ontake argument work out?
We're experiencing the exact same issue, at the time being, and I am curious as to whether you found a solution./Chris
