Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
BertieGG
Frequent Visitor

Dataset pulling from Dataflows running really slow in Service

Hi All,

 

I have a lovely Star Schema model running on a heap of DataFlows which takes about a minute to refresh all the data in the desktop (PBIX) but appears to be taking over an hour to refresh in the workspace.

BertieGG_0-1655990704580.png

 

I've found lots of materials on performance checking the PBIX, and articles about migrating to DataFlow to enhance speed, but nothing about the performance once live.  Any help would be much appreciated.

Thanks in advance

1 ACCEPTED SOLUTION
Icey
Community Support
Community Support

Hi @BertieGG ,

 

Is the Dataset you are refreshing on a Pro, PPU or Premium Capacity Workspace?

 

If you're on a Pro workspace, you are sharing the Pro capacity with all other Pro users on the Microsoft Cloud. You don't see them or their data, and neither do they see you, but you're all using the finite resources Microsoft have put in place to service Pro users. I don't know how many users that is (Microsoft don't release numbers) but in 2016 it was over 5 million. That's why when you schedule a refresh your request is queued and may be run up to 1 hour later. It's not just your refresh that the service is running, but ALL the refreshes requested by ALL the Pro users. (Yes Microsoft splits this up geographically, but we don't know how many are on each Geography, just like we don't know how many in total). Thats why Refreshing on a Pro workspace takes a variable amount of time. Microsoft add servers and improve the service, but more users join and more features are provided that use up Server time. I guess for Microsoft it's a compromise between having millions of Servers standing idle just in case, or Users complaining that Refreshes take more than an hour.

 

Your alternatives are to use a Premium Capacity or Premium Per User (PPU) backed Workspace. Premium Capacity can be thought of as renting your own server from Microsoft in the cloud. It costs more, but you get a dedicated capacity that only you use, so you don't have these service contention problems. PPU offers many benefits of Premium Capacity, but at a lower per-user cost. PPU runs on faster servers and has fewer users, so in many cases will give you a faster refresh.

 

Reference: Solved: Re: (2MB) PBI Refresh via Webservice - Extrem slow - Microsoft Power BI Community

 

 

Best Regards,

Icey

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

2 REPLIES 2
Icey
Community Support
Community Support

Hi @BertieGG ,

 

Is the Dataset you are refreshing on a Pro, PPU or Premium Capacity Workspace?

 

If you're on a Pro workspace, you are sharing the Pro capacity with all other Pro users on the Microsoft Cloud. You don't see them or their data, and neither do they see you, but you're all using the finite resources Microsoft have put in place to service Pro users. I don't know how many users that is (Microsoft don't release numbers) but in 2016 it was over 5 million. That's why when you schedule a refresh your request is queued and may be run up to 1 hour later. It's not just your refresh that the service is running, but ALL the refreshes requested by ALL the Pro users. (Yes Microsoft splits this up geographically, but we don't know how many are on each Geography, just like we don't know how many in total). Thats why Refreshing on a Pro workspace takes a variable amount of time. Microsoft add servers and improve the service, but more users join and more features are provided that use up Server time. I guess for Microsoft it's a compromise between having millions of Servers standing idle just in case, or Users complaining that Refreshes take more than an hour.

 

Your alternatives are to use a Premium Capacity or Premium Per User (PPU) backed Workspace. Premium Capacity can be thought of as renting your own server from Microsoft in the cloud. It costs more, but you get a dedicated capacity that only you use, so you don't have these service contention problems. PPU offers many benefits of Premium Capacity, but at a lower per-user cost. PPU runs on faster servers and has fewer users, so in many cases will give you a faster refresh.

 

Reference: Solved: Re: (2MB) PBI Refresh via Webservice - Extrem slow - Microsoft Power BI Community

 

 

Best Regards,

Icey

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

BertieGG
Frequent Visitor

Many thanks for your response @Icey 

I think you've hit the nail on the head.  I just couldn't seem to find that info when I was looking for it originally.  I had heard about the "Noisy Neighbours" effect, but somehow slipped my mind in this context.

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.

Power BI Header Change

Power BI Header Update

Read about the exciting updates for the Power BI forum.

Top Solution Authors
Top Kudoed Authors