Forum Discussion
Importing models beyond 400 GB
I understand that a semantic model in the Power BI Service can be as large as 400 GB with the F2048 SKU.
Why is this the limit? Is this because there's no difference in performance when compared to DirectQuery? Or is the hardware/software not good enough to handle this size in-memory?
Thanks for your response. While it might seem like adding more RAM and CPU could support larger models, the 400 GB limit in Power BI is not just a hardware issue. It’s a deliberate decision by Microsoft to ensure stability in shared environments.
Here’s the thinking behind it.
1. In Premium capacities, multiple models and users share resources. If one model uses too much memory, it can impact others.
2. Although it’s optimized for speed and compression, very large models can reduce performance and make refreshes and queries slower.3. They take longer to refresh, are more difficult to deploy, and complicate capacity management.
4. Microsoft recommends breaking models into reusable, governed components and using Fabric shortcuts to keep BI solutions scalable and easier to manage.Regards,
Yugandhar.
9 Replies
- FBergamaschi
Super User
bigger and bigger semantic models consumes more and more memory resources, therefore the limitation
Direct Query does not imply memory resources
There is an enormous performance difference between Import (fast) and Directquery (slow)
If this helped, please consider giving kudos and mark as a solution
@me in replies or I'll lose your thread
Want to check your DAX skills? Answer my biweekly DAX challenges on the kubisco Linkedin page
Consider voting this Power BI idea
Francesco Bergamaschi
MBA, M.Eng, M.Econ, Professor of BI
- GilbertQ
I would have to say if you are getting to a 400 gig in memory limit for a semantic model there is something you need to be architect. I have customers who have a billions of rows that are putting into a 75GB semantic model.
V-yubandi-msftCommunity Support
Hi ShadowCircuit87 ,
The 400 GB limit for semantic models in Power BI Service with the F2048 SKU is based on capacity and the use of the Large Semantic Model Storage Format. This format helps models go beyond default limits by storing data more efficiently in memory. The main reason for the limit is resource management and performance stability, as larger models use more memory and can affect overall capacity performance if not managed well.
Import mode loads data into memory using the VertiPaq engine, which is designed for speed and compression, making it generally faster than DirectQuery. In contrast, DirectQuery accesses the source data in real time, which can be slower but doesn’t use memory the same way, avoiding those constraints. So, there’s a clear performance difference between the modes, and the size limit is mainly to ensure reliable performance rather than being solely a hardware restriction.
Regards,
Yugandhar.- V-yubandi-msft
Hi ShadowCircuit87 ,
Could you let me know if your issue is resolved, or if you need any additional details.
Thanks. - V-yubandi-msft
Hi ShadowCircuit87 ,
Please let us know if you are facing any issues so that we will take a look into it and address ASAP.
Thank You. - V-yubandi-msft
Hi ShadowCircuit87 ,
Could you let us know if your issue has been resolved or if you are still experiencing difficulties? Your feedback is valuable to the community and can help others facing similar problems.Hi V-yubandi-msft , couldn't you keep upgrading RAM & CPUs to infinity? (Why is in-memory analytics capped at 400 GB?) I understand there's a performance difference, but are there specific reasons for it?
- V-yubandi-msft
Thanks for your response. While it might seem like adding more RAM and CPU could support larger models, the 400 GB limit in Power BI is not just a hardware issue. It’s a deliberate decision by Microsoft to ensure stability in shared environments.
Here’s the thinking behind it.
1. In Premium capacities, multiple models and users share resources. If one model uses too much memory, it can impact others.
2. Although it’s optimized for speed and compression, very large models can reduce performance and make refreshes and queries slower.3. They take longer to refresh, are more difficult to deploy, and complicate capacity management.
4. Microsoft recommends breaking models into reusable, governed components and using Fabric shortcuts to keep BI solutions scalable and easier to manage.Regards,
Yugandhar.
- V-yubandi-msft
Hi ShadowCircuit87 ,
I wanted to check in regarding your issue. Has it been resolved, or do you need any further information. Let me know if you’d like more details.
Thanks.
