Forum Discussion
Dataset refresh takes too long
Hello all,
I have a Power BI dataset with a single Data Source (Web JSON) and 1 main query to get all the data from the JSON. The JSON isn't very large (less than 100kb) but it does take the web server about 30 seconds to generate and return the JSON.
My dataset has several other queries which reference the main query such that the datasource should only be called once. The majority of these queries remove columns from the main query and then performs some ExpandListColumn and ExpandRecordColumn operations.
I am working in a Premium Workspace and using the Power BI API to refresh the dataset every minute. The Refresh History for the dataset shows that each refresh takes about 2 minutes on average, but sometimes much longer (5 to 10 minutes).
Am I correct in thinking that the 2 minute average Refresh time is due to the Transformations in my Queries that remove and expand columns? Is there any way to optimize these Transformations?
Thank you
5 Replies
The Power BI Service is not dedicating resources exclusively to your refreshes. They can sometimes be queued, and other times run slower.
Requesting new refreshes while an existing refresh is in progress is considered bad practice and should be avoided.
Does your webpage not provide the real time reporting that you seem to require? What's the value add of Power BI?
Hi lbendlin, thanks for the reply.
lbendlin wrote:The Power BI Service is not dedicating resources exclusively to your refreshes. They can sometimes be queued, and other times run slower.
Does this mean there's nothing I can do to improve the Dataset Refresh time other than improve the Data Source load time?
lbendlin wrote:Does your webpage not provide the real time reporting that you seem to require? What's the value add of Power BI?
No, the webpage only serves raw JSON data. Power BI adds value in the form of auto-refreshing visuals representing my data.
Thanks again!
Auto refreshing is something you usually do with Streaming or Push Datasets, or with Dashboards. Have you considered pushing the JSON into a Hybrid Streaming Dataset?
