Forum Discussion

StevenVDP's avatar
StevenVDP
Frequent Visitor
4 years ago
Solved

Long scheduled refresh wait with Pro

Hi there,

 

we currently have set up a scheduled refresh on a single dataset. The Import dataset resides in a Pro workspace and is connecting to Synapse Serverless with a SQL credential.

 

When looking at the refresh history, we notice that the refresh of the dataset takes about +/- 30 minutes but the actual refresh of data should only take 2 or 3 minutes. Looking a bit deeper into the SQL logs, we notice that the Scheduled refresh kicks it on the scheduled hour but the very first SQL query is send from Power BI to Synapse after 25 minutes. In the meantime, Power BI is showing 'In progress' as status in those 25 idle minutes. In other words, it seems like the refresh process is runing idle for quite a while before the actual refresh is happening.

 

Occurences on a time scale (when scheduled refresh was defined at 8 AM):

  • Around 8:00 / 8:03 AM - the scheduled refresh process kicks in
  • From 8:03 AM to +/- 8:25 AM - the status is 'In progress' but when looking at the SQL source, no queries are send from Power BI to Synapse. So this process sits idle for quite a while
  • Around 8:25 AM - we finally see the first queries popping up in SQL
  • Around 8:28 AM - refresh completed

So a 3 minute process takes about 25 minutes.

Is this due to the noisy neighbours concept with Pro licences? Is the process running idle because there are a lot of other refreshes on this exact time point in the shared capacity?

 

What's more, we tested this with a Premium Per User workspace and the scheduled refresh is ready in 3 minutes! So with PPU, the refresh is not running idle but immediately sends out the SQL queries. This evidence makes us think this is fully related to Pro.

 

What are your thoughts? Are we missing something in this story?

 

Need Help
  • Burningsuit's avatar
    Burningsuit
    4 years ago

    Hi StevenVDP 

    For what it's worth, I think you've identfied your problem. This is the "noisy neighbour" scenario and your Pro licence is probably waiting in a queue for its refresh. Especially if you find that moving to PPU speeds the process up (provided you have made no other changes) it seems clear that the culprit is the Pro licence.

    In this scenario I'd try changing the time of the refresh, to see of refreshing earlier or later in the day is faster. Moving refresh to 7:30am may actually get your data refresh done by 8:00am. But on the other hand - why worry ? provided the refresh gets done eventually, you can shift the timeings around to cope and there's no real problem.

    Hope this helps

    Stuart

2 Replies

  • Burningsuit's avatar
    Burningsuit
    Resident Rockstar
    4 years ago

    Hi StevenVDP 

    For what it's worth, I think you've identfied your problem. This is the "noisy neighbour" scenario and your Pro licence is probably waiting in a queue for its refresh. Especially if you find that moving to PPU speeds the process up (provided you have made no other changes) it seems clear that the culprit is the Pro licence.

    In this scenario I'd try changing the time of the refresh, to see of refreshing earlier or later in the day is faster. Moving refresh to 7:30am may actually get your data refresh done by 8:00am. But on the other hand - why worry ? provided the refresh gets done eventually, you can shift the timeings around to cope and there's no real problem.

    Hope this helps

    Stuart

  • Anonymous's avatar
    Anonymous
    Not applicable
    4 years ago

    Hi StevenVDP ,

     

    Agree with Burningsuit .

    Premium Per User (PPU) is a new way to license premium features on a per-user basis, and includes all Power BI Pro license capabilities, along with features like paginated reports, AI, and other capabilities that previously were only available with a Premium capacity. With a PPU license, you do not need a separate Power BI Pro license, as all Pro license capabilities are included in PPU.

     

    Refreshes, like queries, require the model be loaded into memory. If there is insufficient memory, the Power BI service will attempt to evict inactive models, and if this isn't possible (as all models are active), the refresh job is queued. Refreshes are typically CPU-intensive, even more so than queries. 

     

    For more information,please see:How capacities function 

     

    Hope it helps,


    Community Support Team _ Caitlyn

    If this post helps then please consider Accept it as the solution to help the other members find it more quickly.