Forum Discussion
Dataflow intermittent long-running refresh. Not enough detail in logs to thoroughly troubleshoot.
For what it's worth, we have dedicated workspaces for dataflows depending on the subject matter. We don't have any datasets in the same workspace. We only pull the dataflows into our models as needed, and then are published to separate workspaces.
Power ParticipantYes that is good management of Workspaces in my opinion, but I'd still run a copy of that Dataflow independently in it's own Workspace for a few days and see what happens. Scheduling it to run concurrently with the original could give an interesting result. If the test Dataflow refreshes quickly at times when the original lags, it may point to blocking within the original Workspace.
You'd think it's either your Source prioritizing inbound queries or BI Service prioritizing Dataflows, I believe you've confirmed the first thing is not happening.
I will try that. Thank you for the suggestion.
The primary complaint is that the Service and logs fail to provide sufficient information to easily determine the root cause. If it's a prioritization issue, it would be nice to see that detailed out somewhere. Running these tests may not lead to a definitive answer, which puts me right back at square one. 😑- cassidy
Oh I hear you. A lot of BI Service is "magic optimization", which both allows low experience folks to do great things ( me ), but at the same time will not reveal what's really happening (especially around refreshing).
For example, Dataset refreshes are also like a black box, except at least one helpful person figured out how to analyze the progress and display bottlenecks https://dax.tips/2021/02/15/visualise-your-power-bi-refresh/