Power BI Out of Memory with Incremental Refresh - large dataset

[FatimaSafa]

Hi,
I am working with a very large dataset for a hospital project in Power BI. The data goes from 2019 until today (~25,000 rows per week), and I enabled Incremental Refresh using PostedOn as the datetime field. I set RangeStart = 01/01/2019 and RangeEnd = 31/12/2100, and configured the refresh to store 6 years and refresh only the last 5 days.

However, during the first refresh in Power BI Service, it crashes with Out of Memory. I assume it's due to the initial full load of millions of rows.

I need all data from 2019 to now in one single dashboard, with daily automatic updates — without splitting reports or using Premium.

Any advice to optimize this setup or workarounds that can help me stay within Pro limits?

Thanks!

[v-tsaipranay]

Hi @FatimaSafa ,

We haven’t received an update from you in some time. Could you please let us know if the issue has been resolved?
If you still require support, please let us know, we are happy to assist you.

Thank you.

[v-tsaipranay]

Hi @FatimaSafa ,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions.

Thank you.

[v-tsaipranay · Accepted solution]

Hi @FatimaSafa ,

Thank you for confirming the steps you've taken. It's clear you've implemented the bootstrapping approach correctly, and reducing the volume in data_25 is a good optimization.

The new error, DM_GWPipeline_Client_AsyncOperationExpired, indicates a query timeout rather than a memory issue. This usually occurs when a refresh takes too long, especially with multidimensional cube sources that may not support efficient query folding. Even if there are fewer rows, complex merges or transformations in Power Query can still slow things down.

To resolve this, try simplifying any transformations in data_25, avoid complex merges on cube data, and check if your queries are folding as expected. If necessary, consider pre-processing logic with dataflows or splitting data_25 into smaller parts, such as by quarter. It may also help to review gateway performance, as outdated or underpowered gateways can cause this issue. 

Hope this helps. If you need any further assistance, please feel free to reach out.

Thank you.

[v-tsaipranay]

Hi @FatimaSafa ,

I wanted to follow up on our previous suggestions. We would like to hear back from you to ensure we can assist you further.

Thank you.

[FatimaSafa]

Hi @v-tsaipranay 

Thank you for following up.

I’ve completed the configuration as suggested. Specifically, I’ve bootstrapped the datasets data_192021, data_222324, and data_25, merged them accordingly, and disabled both load and refresh for the first two (data_192021 and data_222324). Only data_25 remains active for refresh.

Please note that the source is a multidimensional cube with multiple dimensions and measures.

This is the new error I'm encountering during data refresh:

Data source error: {"error":{"code":"DM_GWPipeline_Client_AsyncOperationExpired","pbi.error":{"code":"DM_GWPipeline_Client_AsyncOperationExpired","parameters":{},"details":[],"exceptionCulprit":1}}}

Also note that the data from 2025 until today contains only around 700,000 rows, so the volume itself doesn’t seem excessive.

[v-tsaipranay]

Hi @FatimaSafa ,

Thank you for reaching out to the Microsoft Fabric Community Forum. Also thank you @lbendlin  for your response.

Based on your scenario, the Out of Memory error during the initial incremental refresh in Power BI Service is expected, especially under Power BI Pro. This is because the first refresh loads all historical data since 2019, and with millions of rows, it can easily exceed the 1 GB compressed dataset limit allowed in Pro.

As correctly mentioned by @rohit1991 , Power BI Pro is not optimized for large datasets unless the model is significantly trimmed or pre-processed.

To address this within your existing license, we recommend a dataflow-based bootstrapping approach. First, move all historical data from 2019 up to a recent cutoff (e.g., last month) into a Power BI Dataflow. This shifts the heavy load out of the dataset. Then, in your Power BI Desktop model, connect to that dataflow and configure Incremental Refresh only for recent data (e.g., the last 1 year).

This significantly reduces memory consumption during refresh while keeping your report fully unified and up to date. Additionally, optimize your model by removing unused columns, reducing cardinality, and aggregating data where possible.

This setup allows you to maintain a single report with full data coverage and daily refreshes, all within Power BI Pro limits. Let us know if you'd like help implementing this pattern.

You can also refer to the official incremental refresh guidance for further details.

Hope this helps. Please reach out for further assistance.

Thank you.

[rohit1991]

Hi @FatimaSafa ,

With Power BI Pro, your dataset size is limited (1 GB compressed), and the first incremental refresh always loads all historical data at once, which often causes “Out of Memory” errors on large datasets.

How to optimize or work around: Remove unused columns and pre-aggregate your data if possible. Reduce the data range (e.g., load only recent years or months if business allows). Use Power BI Dataflows: Load historical data into a dataflow, then keep your dataset (with incremental refresh) focused on recent data. Upgrade to Premium Per User (PPU) if you truly need all data together and can’t shrink the model.

Bottom line you’ll need to trim the dataset size or use dataflows for older data. Power BI Pro isn’t designed for large historical datasets with millions of rows unless you reduce the size significantly.

[lbendlin]

One word - bootstrapping.

Troubleshoot incremental refresh and real-time data - Power BI | Microsoft Learn

Partition limit is 10GB in Premium/Fabric SKUs, timeout limit is 5 hrs per refresh.