Forum Discussion
Incremental Refresh Policy in Dataflows and Datasets
Hi, friends!
I am working with a big table (more than 100 million rows), and to increase the performance, I am configuring the incremental refresh. My structure is:
- Data is extracted from a DB2 database using a dataflow; and,
- Data is loaded into a dataset in Power BI to generate my reports.
I am only setting the incremental refresh policy at the dataflow. However, I would like to hear about your experiences. Is it necessary to apply the same policy in the dataset?
3 Replies
It just depends on what you need performance wise.
Incremental on a Dataflow reduces pressure on the Source, Power BI Service, and reduces Dataflow run time.
- If you then sourced that Dataflow in Desktop and made that Incremental too, you would reduce pressure on Power BI Service and Dataset refresh run time.
For me, Incremental refresh on Desktop is a absolute no-go because you lose the ability to download the Dataset from Service.
If your Dataset refresh time is acceptable to you, I wouldn't turn on incremental as it has downsides along with the upsides.
We use often incremental refresh for dataflows as well for datasets.
Maybe you can't download it (but you can by using external tool) but why there is a need to download this file when you're the developer of this report? In other cases I hope you have all organizational .pbix files in a place where you can pic the last version.
Regards
The main reasons for me:
- Downloading from Service guarantees that any developer will get the latest version.
- Downloading from Service already has the latest data loaded, making quick changes even quicker by not having to refresh locally.
- Having the option to download from Service is a another backup source incase of failures with our other storage options.
