Data Latency issue for on premises HANA source to dataset refresh in Power BI service

[vkarumanchi]

Dear Team,

I have a on premises HANA data source and facing issues in data incremental refresh. Here are the different refreshing and its respective times to complete the refresh for same amount of data

• For getting data to desktop file took 7 min
• For on demand refresh is taking 15 min
• Scheduled refresh took 29 min (Using On-Premises data gateway configaration)

Please suggest me how to chech WHY and WHERE is latency is happening.

Thanks,

Vara

[Anonymous · Accepted solution]

HI @vkarumanchi,

Perhaps it should be related to the refreshing schedule of the backend processing.
For power bi desktop refresh, it directly connects to your data source, so it will be refreshed quickly if your networks and data source not too far away and busy to access.

For on-demand refresh and schedule refresh, they required to use azure data cent and data busy to handle the refresh. They may be effect by a few reasons. (e.g. networks status, service idle resources, network delay of request send/receive between different data region and servers, queue await of refresh plan...)

You can take a look at the following documents to know more about them.

Data refresh in Power BI 

Configure scheduled refresh#scheduled-refresh 

Power BI Security#power-bi-architecture 

On-premises data gateway architecture#how-the-gateway-works 

Regards,

Xiaoxin Sheng

[lbendlin]

On-Demand and Scheduled Reporting use the same gateway.  Maybe your gateway machines are not up to the task. You want lots of memory on each of the cluster members (32GB if possible), lots of cores (8 if possible) and lots of free and fast disk space (equivalent to the P SKU Memory size) in addition to a nice thick network pipe.  Consult the gateway logs for more precise timing and actual load distribution.

Note: If anyone has a good template to ingest the gateway log files - I would very much like to see that.