Forum Discussion
Helper IVIncremental data load in Power BI file
Hi Marico ,
Yes, you can improve report performance even with datasets larger than 5 GB. Power BI offers several methods to help optimize speed and user experience. Here are some tips:
- Use Aggregations: Create summary tables to reduce data volume and improve load times. Power BI can switch between aggregated and detailed data as needed.
- Optimize the Data Model: Remove unused columns and tables, avoid high-cardinality columns, and use numeric keys for relationships.
- Use Hybrid or Composite Models: Store large fact tables in DirectQuery mode and import only smaller dimension tables to save memory.
- Apply Incremental Refresh: Use incremental refresh for large datasets with date/time fields to process only recent or changed data.
- Limit Visual and Filter Interactions: Reduce the number of slicers and filters, and use the "Apply" button for filters to control when visuals update.
- Split Reports: Divide complex reports into multiple pages or files so users only load what they need.
- Use Performance Analyzer: Identify slow visuals or queries with the Performance Analyzer and optimize them.
- Upgrade to Power BI Premium: Premium capacity provides more memory and processing power for large datasets.
Thank you again.
Community SupportHello Marico ,
Thanks for reaching out to the Microsoft fabric community forum.
Yes, you can achieve this in Power BI using the Incremental Refresh feature. This is particularly helpful for large datasets like yours with 70–80 million rows. Incremental Refresh lets you update only new or changed data based on a date column, rather than refreshing the entire dataset each time, which boosts performance and saves time. You set this up in Power BI Desktop, but keep in mind that incremental loading only works after publishing to the Power BI Service. In Desktop, it still does a full refresh for previewing.
To set it up, create two DateTime parameters in Power Query: RangeStart and RangeEnd. Filter your date column using these parameters, then load the data. Right-click your table and choose “Incremental Refresh” to define how much data to keep and how often to refresh. After publishing to Power BI Service, the refreshes will run as configured.
This feature is available in Power BI Pro, Premium Per User (PPU), or Premium capacity workspaces. Ensure your date column is in Date/Time format and properly indexed for best performance. If these requirements are met, Incremental Refresh should work effectively for your large dataset.
If you need detailed setup instructions, feel free to ask.
Thank you,
Tejaswi.
Helper IVCan you share detailed steps with screenshots and a sample file?
- v-tejrama
Hi again Marico ,
After reviewing Microsoft’s official documentation on Incremental Refresh in Power BI and the community post at Community Post on Incremental Refresh, I can confirm that Incremental Refresh can absolutely be implemented using a .pbix file. There are a few key steps to follow:
Set up the RangeStart and RangeEnd parameters in Power Query.
Apply filters with these parameters to a date column in your fact table.
Once published to Power BI Service, configure the Incremental Refresh policy in the dataset settings. The initial refresh will process all historical data, but subsequent refreshes will only update the latest partition as specified (such as the last 7 days).
I have shared the incremental document link, the Microsoft link, and also included screenshots. Please review them.
Incremental refresh for semantic models in Power BI - Power BI | Microsoft Learn
Solved: Incremental Load is a Power BI - Microsoft Fabric Community
Thank you.- Marico
v-tejrama Thanks for sharing this.
However, my problem is different. My concern is regarding the incremental refresh will help to refresh the data quickly. However, it won't help to access the report quickly when client applying the filter on the report.
At the moment, even one filter applied on the report page takes 5 mins to bring the data to screen. Report is super slow right now in service.
I am currently using import mode. Will direct mode or any other way help the report accessibility quickly?
- v-tejrama
Hi Marico ,
yes, once you’ve set up the Range Start and Range End parameters in Power Query and applied them to a date column, the Incremental Refresh works exactly as intended in Power BI Service. When you first publish the report, it performs a full refresh to load all historical data and establish the required partitions. After that, during scheduled or manual refreshes, it only processes the latest data based on the policy you've defined (like the last 7 days), rather than reloading the entire dataset again. This improves performance and efficiency. If you're noticing a full dataset refresh every time, it’s worth checking if the parameters are properly applied in your date filter, and whether the dataset was refreshed at least once post-publish to create those partitions. I’ve also shared Microsoft documentation and screenshots earlier to support this setup.
Thank you .