PBI Gateway extremely slow - what can we do

[sci66]

We want to pull data from PowerBI service connecting to AWS Athena table. We have AWS EC2 running gateway. We have succesfully created gateway connection and it shows up in PowerBI Service. We are able to do data pull of small amount however it is extremely slow. As soon as we get to around 500k records, it keeps spinning and nothing loads up. 

Has anyone encountered something similar, how can we make this work so that we can pull large amount of data. (in millions)

We have looked into EC2 and when we run large pulls it is not showing any large resouce hence this is not bottleneck. I talked to our AWS archtecture and they say that gateway Ec2 & AWS Athena are in same zone. 

I am not sure at this point if this is because Powerbi service (running in Azure)  is pulling via AWS is causing this. 

Any help, any insight appreciated. thanks. 

[lbendlin]

Is your EC2 server connected to your company network via VPN?  That will cause excessive amounts of to-and-fro network traffic.  Gateway VMs are supposed to be on-prem, ideally close-ish to the data sources.

[v-yingjl]

Hi @sci66 ,

If you've fixed the issue on your own please kindly share your solution. If the above posts help, please kindly mark it as a solution to help others find it more quickly. Thanks!

Best Regards,
Yingjie Li

[v-yingjl]

Hi @sci66 ,

In Power BI Service, it depends on several factors:

1. The size of data model in the dataset.
2. The complexity of data model.
3. The network connection between Gateway and data source.
4. The network connection between the Gateway and Power BI Service.
5. CPU & Memory resource in the machine that installing gateway.

You can refer this blog and document about optimize gateway:

1. power-bi-gateway-infographic 
2. Monitor and optimize on-premises data gateway performance 

Best Regards,
Yingjie Li

If this post helps then please consider Accept it as the solution to help the other members find it more quickly.

[lbendlin]

make sure your gateway has lots of RAM (32GB+ preferred)

make sure your gateway has plenty of free disk space

make sure your gateway has fast disks.

make sure your gateway's swap file is not on the same disk as the gateway files.

Use gateway clusters if you can.

[catcollector]

@lbendlin May I ask where I can check these items you've listed above? I am encoutering similar issue as the OP.

We have an EC2 server where an On-Premises Gateway is installed. We've also installed Athena driver and configured it. When our business partners create a dataflow connecting to AWS Athena, then refresh to load data, it took 30 minutes to load 302k records. When we query the same table in Athena, it takes mere seconds. TIA!