Forum Discussion
Helper IIIUnderstanding Gen1 Dataflow Refresh
Hi vamshikrishna20 ,
Power BI incremental refresh will only re-process partitions where data has changed, or those that failed in a previous run. When you cancel a refresh after it’s already loaded some partitions, Power BI doesn’t roll everything back it will pick up from the last successful partition next time, only reloading the ones that need it.
So after a cancelled refresh, your next refresh is typically incremental and much faster, which is why you only saw the latest partitions in the logs. This is normal behavior Power BI is designed to save time and resources by not reloading data it already ingested.
If you ever need a true full refresh (reload all partitions from scratch), you’d have to change the incremental refresh policy or delete and re-create the dataflow. But for most cases, letting Power BI handle it this way is fine.
For more insight, check the refresh history and the CSV logs to see exactly which partitions loaded and how many rows were written.
Helper IIn the event it didn't load all your data into partitions properly.
Have you looked into the following documentation from Oracle Performance Tuning for Large Data Retrievals
I found adjusting the driver registry settings on the gateway helped when it got to a certain rowsize that timed out.
However, in the end the only solution was to have smaller partititions (shrinking from years to quarters) connecting via SQL analysis services and manually building one partition at a time to get my data ingested.