Forum Discussion
Power Query load from Dataflow is excruciatingly slow
- 3 years ago
Try clearing the cache in Power BI Desktop.
If that doesn't help, is this on a laptop? Go to Starbucks or someplace else and see if the internet there is better. if that works, there could be a routing issue from your location that is causing a problem, or some other corporate firewall.
Also, you don't have to wait for the preview to finish to load. Just close Power Query and tell it to save (and apply) the data. Even if it fails the first time, hitting Refresh in Power BI desktop usually clears that up.Edit: oh, what dataflow connector are you using? Does the Source line say PowerPlatform.DataFlow or PowerBI.dataflow? Whatever it says, switch it out. I personally prefer the PowerBI.Dataflow as for raw speed it is faster, but doesn't allow folding or setting incremental refresh with it.
Great. If you do test the Power BI connector and it works better, and you need help swapping out the M code, let us know. It is a 30 second code swap in the Advanced Editor with no need to redo any other transformations.
In the mean time, could you mark one or more as the solution that at least guided you on this so this thread can be marked as closed? Thanks, and have a good weekend!
Could you outline what you'd call the 30 second switch between Power Platform dataflows and PowerBI dataflows.
I'm still having issues with this (have been focused on my AV) which I thought was causing the issue. haven't got to the cache clearing and remapping the dataflows yet.
- edhans3 years agoCommunity Champion
Go into Power query:
- Create a new dataflow connection using Power BI dataflows to that same table.
- That will generate 3-4 steps, a SOURCE, and a few Navigation steps.
- Open the Advanced Editor and copy those first few lines. Do NOT copy the "in" that will be on line 4 or 5
- Go to your original and paste over those same first few lines. (make a backup of that code obviously)
- At the end of the last line you copied in, you need to add a comma. It was the last line in the original so had no comma.
- Line 5 (or whatever the first line of the original dataflow you didn't copy over, just make sure the table reference refers to the name of the step before. It will likely be the name of the table but without the underscore.
So say this is your original Power Platform dataflow:
let Source = PowerPlatform.Dataflows(null), Workspaces = Source{[Id="Workspaces"]}[Data], #"9c50e50c-bbe6-4c3a-b6cd-b2623250536c" = Workspaces{[workspaceId="9c50e50c-bbe6-4c3a-b6cd-b2623250536c"]}[Data], #"d4c8ee27-89ac-40b7-958f-c3e8e7c69759" = #"9c50e50c-bbe6-4c3a-b6cd-b2623250536c"{[dataflowId="d4c8ee27-89ac-40b7-958f-c3e8e7c69759"]}[Data], Products_ = #"d4c8ee27-89ac-40b7-958f-c3e8e7c69759"{[entity="Products",version=""]}[Data], #"Filtered Rows" = Table.SelectRows(Products_, each [ProductKey] > 1800) in #"Filtered Rows"This is the same dataflow but withthe Power BI dataflow:
let Source = PowerBI.Dataflows(null), #"9c50e50c-bbe6-4c3a-b6cd-b2623250536c" = Source{[workspaceId="9c50e50c-bbe6-4c3a-b6cd-b2623250536c"]}[Data], #"d4c8ee27-89ac-40b7-958f-c3e8e7c69759" = #"9c50e50c-bbe6-4c3a-b6cd-b2623250536c"{[dataflowId="d4c8ee27-89ac-40b7-958f-c3e8e7c69759"]}[Data], Products1 = #"d4c8ee27-89ac-40b7-958f-c3e8e7c69759"{[entity="Products"]}[Data] in Products1So I will copy the Source through Products1 lines into my original and it becomes this:
let Source = PowerBI.Dataflows(null), #"9c50e50c-bbe6-4c3a-b6cd-b2623250536c" = Source{[workspaceId="9c50e50c-bbe6-4c3a-b6cd-b2623250536c"]}[Data], #"d4c8ee27-89ac-40b7-958f-c3e8e7c69759" = #"9c50e50c-bbe6-4c3a-b6cd-b2623250536c"{[dataflowId="d4c8ee27-89ac-40b7-958f-c3e8e7c69759"]}[Data], Products1 = #"d4c8ee27-89ac-40b7-958f-c3e8e7c69759"{[entity="Products"]}[Data], #"Filtered Rows" = Table.SelectRows(Products1, each [ProductKey] > 1800) in #"Filtered Rows"That is it. All subsequent steps (Filtered Rows in my example) remain unchanged.
5min tops.
Don't forget to delete the newly created dataflow you used to generate the code, or it will load that into the model too. You just needed it to get the code.
- jnickell3 years agoHelper V
Thanks, was wondering if you had an approach of easily identifying what to tweak in the existing query, without copy/paste. The copy/paste approach works and isn't bad, just a little tedious.
Just finished remapping each of the dataflows in the model. Still waiting to see if that improves things.