Forum Discussion
VNET Gateway and max dataset size
Hi,
I plan to switch from on premise gateways to vnet gateway for my project (to save costs of course).
I heard that there are some limitations regarding dataset size that can be supported but no-one knows details.
I know that Gateway has fixed hardware of 2 cores, 8 GB RAM and it cannot be improved but what does it mean?
I have few import-mode dataset 5-10 gb each (according to daxstudio).
Does it mean that I cannot use datasets bigger than 8 gb? or bigger than like 5-6gb?
Does multiple connections to 3-5 gb datasets also can cause the performance issue? let's say 6 connections to 5gb datasets
thank you,
Kuba
Hi Cyferki , Thank you for reaching out to the Microsoft Community Forum.
The 8 GB RAM limit applies per query execution, not total dataset size. A 5–10 GB dataset (compressed in Power BI) can temporarily expand 3–4x during refresh, potentially requiring 30-40 GB RAM, which exceeds the gateway’s capacity. Datasets larger than 5-6 GB (compressed) are likely to fail during refresh due to memory exhaustion. If multiple datasets refresh at similar times, they compete for memory and CPU, leading to performance degradation or failures. VNet Gateway does not support clustering, so you cannot scale out by adding more nodes.
You can use aggregations, remove unnecessary columns, or split large tables. Use DirectQuery or Composite Mode instead of full Import Mode to reduce memory usage and improve scalability when working with large datasets. If large Import Mode datasets are required, an on-premises gateway (with clustering) is a better choice.
Please refer to below documentation for more information:
Virtual network data gateways capacity consumption | Microsoft Learn
Manage virtual network (VNet) data gateway high availability and load balancing | Microsoft Learn
Validate VPN throughput to a virtual network - Azure VPN Gateway | Microsoft Learn
Validate VPN throughput to a virtual network - Azure VPN Gateway | Microsoft Learn
If this helped solve the issue, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details, always happy to help.
Thank you.
3 Replies
I was not aware of these resource constraints on VNet gateways. Can you point to the documentation?
Gateway configurations should follow the expected workloads. For Direct Query you need predominantly a fast network connection and lots of cores. For Import Mode mashups you predominantly need lots of memory (8GB is inadequate, 32 GB is better) and a fast storage system (ideally not on the system drive). At least 50 GB, better more.
I don't think you can save much cost with a VNet gateway compared to a couple of on-prem VMs with 16 core, 32 GB RAM, 10Gbps Ethernet and nothing else running on them.
Hi lbendlin , Thank you for your valuable insights. Please refer below documentation regarding the aforementioned resource constraints:
The Fabric throttling policy
Hi Cyferki , Thank you for reaching out to the Microsoft Community Forum.
The 8 GB RAM limit applies per query execution, not total dataset size. A 5–10 GB dataset (compressed in Power BI) can temporarily expand 3–4x during refresh, potentially requiring 30-40 GB RAM, which exceeds the gateway’s capacity. Datasets larger than 5-6 GB (compressed) are likely to fail during refresh due to memory exhaustion. If multiple datasets refresh at similar times, they compete for memory and CPU, leading to performance degradation or failures. VNet Gateway does not support clustering, so you cannot scale out by adding more nodes.
You can use aggregations, remove unnecessary columns, or split large tables. Use DirectQuery or Composite Mode instead of full Import Mode to reduce memory usage and improve scalability when working with large datasets. If large Import Mode datasets are required, an on-premises gateway (with clustering) is a better choice.
Please refer to below documentation for more information:
Virtual network data gateways capacity consumption | Microsoft Learn
Manage virtual network (VNet) data gateway high availability and load balancing | Microsoft Learn
Validate VPN throughput to a virtual network - Azure VPN Gateway | Microsoft Learn
Validate VPN throughput to a virtual network - Azure VPN Gateway | Microsoft Learn
If this helped solve the issue, please consider marking it 'Accept as Solution' so others with similar queries may find it more easily. If not, please share the details, always happy to help.
Thank you.
