Daily refresh times

[Anonymous]

We have a lot of large data sets that need updating daily. Is it better to set the refresh to daily without a time specified f, or specify a time and stagger them through the day?

Note that there are a lot of other datasets from other business areas using the service throughout the day.

[Anonymous]

Hi  @Anonymous  ,

You set it to the same time, theoretically it will be executed in parallel at the same time, but each report will have a certain time delay according to the current network status and the size of the data set, such as 15 minutes to 1 hour.

If you perform parallel execution at the same time, it depends on whether your capacity is sufficient. When the report is refreshed at the same time, the model used for refreshing will be loaded. Once there are a lot of models in the memory, the memory will be full, so that the model generated by the subsequent refresh operation will not be loaded and enter the queued state. The report will be slow to load or even fail

You can check the refresh time by viewing the Refresh Summary and metric usage app to make reasonable allocations. Try to choose free time for refreshing, and divide the refreshing time that is concentrated and refreshing together, so as to avoid too many people to occupy resources and cause queuing

For related content, you can check this link:

https://docs.microsoft.com/en-us/power-bi/admin/service-premium-what-is

Best Regards,

Liu Yang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous]

Thanks. Regarding your comment "Once there are a lot of models in the memory, the memory will be full, so that the model generated by the subsequent refresh operation will not be loaded and enter the queued state. The report will be slow to load or even fail"

How can we ensure that queued refreshes do not fail?

[Anonymous · Accepted solution]

Hi  @Anonymous  ,

1. You can view Power bi Premium Capacity Metrics to view the number of refreshes in a day's time period. By viewing the length of the bar graph, you can determine which time period has more refresh times and which time period has less refresh times. Then you can manually schedule the refresh time by yourself (accurately set refresh in a certain time period of the day), so that you can balance it

2. Try to refresh the report when no one is using the report, such as in the early morning, because the report will also occupy memory when it is interacting.

About Power bi Premium Capacity Metrics and How capacities function, you can check the following link:

https://docs.microsoft.com/en-us/power-bi/admin/service-premium-metrics-app

https://docs.microsoft.com/en-us/power-bi/admin/service-admin-premium-monitor-capacity

https://docs.microsoft.com/en-us/power-bi/admin/service-premium-what-is#power-bi-premium-generation-2-preview

Best Regards,

Liu Yang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous]

Hi Liu Yang,

You say to "set it to the same time", but as you can see in the screenshot below a daily refresh can be set without setting a time. Indeed, specifying a time seems to be causing problems.

Can I clarify your suggestion with an example?

- Our Premium capacity allows 180 minutes of refresh every half hour, 8,640 refresh minutes per day. 

- We have 200 datasets that in total take 5,000 refresh minutes.

- The datasets are significantly different in size, some take only 5 minutes to refresh, others up to 180 minutes.

- These only need to be refreshed daily.

Your suggestion is that we need to manually schedule the refreshes ourselves, calculating the refresh time for each 30-minute period and adding datasets accordingly. 

 Surely this is something that the premium capacity should be expected to manage?

 Can this be clarified with the product team?

[Anand24]

Hi @Anonymous ,

For automatic daily refresh, you have to set the time compulsorily.

The best way in my opinion would be to set the refresh daily after ETL load completes. It would be best if this ETL load time and subsequent Power BI dataset refresh times are in non-business hours.

Let's say your ETL load starts at 4:00 AM EST and completes by 4:40AM EST, then you can keep your Power BI refresh at 5:00AM EST.

You can also schedule the data refreshes after every 2 hours or so with 8 scheduled refreshes permitted with PRO License.

Give a Thumbs Up if this post helped you in any way and Mark This Post as Solution if it solved your query !!! Proud To Be a Super User !!! LinkedIn

[Anonymous]

Hi, in the dataset settings > scheduled refresh, we are able to set daily refresh without specifying a time. Please see screenshot showing refresh settings (note no time specified) and refresh history (showing daily refreshes soon after midnight). 

[Anand24]

@Anonymous ,

That's indirectly an automatic refresh for 12:00AM.

Give a Thumbs Up if this post helped you in any way and Mark This Post as Solution if it solved your query !!! Proud To Be a Super User !!! LinkedIn

[Anonymous]

The question is then, if a lot of datasets are set to daily without a specified time, will the service 'queue' them so they run until all are complete, even if it takes several hours?

Or, will it try to run them all at midnight, and if there is not sufficient refresh duration in that half hour slot, fail the refresh?

I am hoping that the former is correct!