Gateway Low CPU and Memory Usage

[vitorp]

Hi, my gateway is installed inside a VM with 36 vCPUs and 233GB of ram, there is nothing installed in this server, just my gateway.

The gateway is using 1~3% of CPU and about 2~4GB, i changed some parameters inside the configuration file, but nothing has changed. Not even my projects are more fast or slow, they are all taking the same time to run.

My projects are connected to a Postgres (i'm trying to implement an ETL with a SQL Server DW, but this is what i got to work with now).

I want to increase CPU and Memory Usage in order to gain performance and lower my projects refresh time, can anyone help?

Some of my settings:

<Microsoft.PowerBI.DataMovement.Pipeline.GatewayCore.GatewayCoreSettings>
<setting name="DbConnectionPoolMinSize" serializeAs="String">
<value>256</value>
</setting>
<setting name="DbConnectionPoolMaxSize" serializeAs="String">
<value>2048</value>
</setting>
<setting name="DbConnectionPoolExpirationTimeout2" serializeAs="String">
<value>00:10:00</value>
</setting>
<setting name="DbConnectionStatefulPoolMaxSize" serializeAs="String">
<value>2048</value>
</setting>
<setting name="DbConnectionStatefulPoolExpirationTimeout" serializeAs="String">
<value>00:10:00</value>
</setting>
<setting name="SpoolerOperationMaxOutstandingStreamedPackets"
serializeAs="String">
<value>10000</value>
<setting name="StreamBeforeRequestCompletes" serializeAs="String">
<value>True</value>
</setting>
<setting name="MashupDQPoolContainerMaxCount" serializeAs="String">
<value>10000</value>
</setting>
</setting>
<setting name="MashupTestConnectionPoolContainerMaxInstanceCount"
serializeAs="String">
<value>10000</value>
</setting>
<setting name="MashupDefaultPoolContainerMaxCount" serializeAs="String">
<value>10000</value>
</setting>
<setting name="MashupAzureConnectorsCachingPoolContainerMaxCount"
serializeAs="String">
<value>10000</value>
</setting>
<setting name="PowerQueryOnlineCachingPoolContainerMaxCount"
serializeAs="String">
<value>10000</value>
</setting>
<setting name="MashupDisableContainerAutoConfig" serializeAs="String">
<value>True</value>
<setting name="QueryExecutionAggregationTimeInMinutes" serializeAs="String">
<value>5</value>
</setting>
<setting name="SystemCounterAggregationTimeInMinutes" serializeAs="String">
<value>5</value>
</setting>
<setting name="EnableFastCombine" serializeAs="String">
<value>False</value>
<setting name="CPUUtilizationPercentageThreshold" serializeAs="String">
<value>0</value>
</setting>
<setting name="MemoryUtilizationPercentageThreshold" serializeAs="String">
<value>0</value>
<setting name="MashupDefaultPoolContainerMaxWorkingSetInMB" serializeAs="String">
<value>32000</value>
<setting name="BypassConcurrentOperationLimit" serializeAs="String">
<value>True</value>

[GilbertQ]

Hi @Anonymous 

I have to disagree with your statement, if you have a lot of Power Query transformation steps that will consume CPU on the Gateway Server.

Also depending on how many mashup containers you have configured to run in paralell and the size of each mashup container will cause an increase in memory on the gateway server.

The only time this wont be affected is if it is using DirectQuery or LiveConnection.

[Anonymous]

Hi @GilbertQ ，

Yes! You're right!

I am sorry for my misunderstanding and thank you very much for making this clear to me😊.

Best regards,

Community Support Team Selina zhu

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous]

Hi @vitorp ，

In fact, the CPU and memory usage of the gateway does not have much effect on the refresh performance.

Generally, refreshing is easily affected by the network environment, the sheer volume of data, capacity limitations, etc.

You can consider the following recommendations to establish and maintain reliable data refresh processes for your datasets:

Related link:

Data refresh in Power BI - Power BI | Microsoft Docs

Best regards,

Community Support Team Selina zhu

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[GilbertQ · Accepted solution]

Hi @vitorp 

I think to make it faster you need to get your Postgres server to return the queries faster. 

The Gateway is just a throughput device between your Postgres server and the Power BI Service. I would suggest chatting to your DBA to see how they can increase query performance and through put.