Forum Discussion
Queries over large data tables
jPinhao I think in the last SU we turned on parallel loading of queries. What we didn't realize is in some context it actually hurts the performance. I believe we can add an UI option to go back to sequential loading. I'll follow up with the BI team internally about this.
Thanks for the info pqian. For what it;s worth, I was having similar performance issues with the previous version (February update).
It might be nice having that control over linear / parallel loading, another option would be for Power BI to be able to predict memory usage and better manage execution mode and resource usage (although this would undoubtely be more complex, if even possible in some cases).
On the note of referenced queries seemingly running repeatedly as part of their users, can you confirm whether this is the case? Would this be a by-product of parallel evaluation, or has it always been the case with Power BI? Is there a way to prevent that from happening?
jPinhao Referenced queries will execute once per query loaded, so it could be possible that they will hit the data source multiple times - in parallel loading cases they certainly will. It's a function of how each referencing queries ended up folding to the source and how the data is been cached.
For a more detailed explanation, see this post:
While we are discussing options to turn off parallel processing with the teams involved, there's a workaround for your scenario. You can set an environment variable in the PBIDesktop.exe process to go back to sequential loading.
For example, in the command prompt, you can:
> SET PBI_EnableMultithreadedDataLoad=0
> PBIDesktop.exe
Thanks pqian!
This was quite helpul to better follow the execution order, and even helped us pinpoint a few inneficiencies in our queries :) As it stands I don't think what we're trying to do with Power BI is quite possible, but it helps to know we do need an extra layer to process our data!