Forum Discussion
PBI Dataset Not Refreshed Due to Reaching Memory Limit
- 2 years ago
Ensure you have Large Dataset turned on for the dataset sirahcy , and see this article. It specifically mentions a 12GB dataset in a P1 capacity might cause issues, but a 9GB may depending on what else is loaded at the time. Large datasets in Power BI Premium - Power BI | Microsoft Learn
Are you using incremental refresh? The entire dataset does not need to be loaded during refresh if you are using incremental refresh on the large fact table(s). Only those recent partitions that would refresh. - 2 years ago
Thank you edhans. Yes. We're enabled for Large dataset and incrementl refresh on the large fact tables. Some of the datasets actually approached 12GB or exceeded so we definitely will look into sizing the capacity. We're using incremental refresh (e.g. ~60% of a fact table), but not sure how much saving in memory usage, instead of 2x dataset size using full refresh.
We tend to upscale the fabric instance before a refresh, supplying us with enough memory for multiple dataset refreshs. Afterwards we will downscale again. I don't understand why there are such low memory limitations in the first place though.
- sirahcy2 years agoFrequent Visitor
Thanks. Will discuss with admins to enable this setting.