Forum Discussion
Gen2 Dataflow to Warehouse - really slow performance
Hi Scott,
It's difficult to tell what could be the reason for this performance. It could be the gateway or the computer running the gateway. The data source having some bottlenecks or perhaps something on the Fabric service end (perhaps your capacity or something different).
The best way to troubleshoot this would be to raise a support ticket and have an engineer take a closer look at this performance.
Will do - thanks miguel . Just so odd that this takes seconds (which is what I expect) in a Lakehouse but then drags out so long in a warehouse. I'll get an easily reproducible case ready, will update this thread if I'm able to narrow down the issue with support.
Thanks!
Scott
I'm experiencing the exact same thing as you. My DFG2 takes a long time to write to the warehouse, but only takes a fraction of the time to write to the lakehouse. Hopefully support has a solution for you, but I modified my process to use a Pipeline instead:
- Refresh the dataflow (destination: lakehouse)
- SQL Script to delete the data from the warehouse (beneficial for incremental data loads)
- Copy data from the lakehouse to the warehouse.
This process is much quicker than when I was trying to write directly to the warehouse.
I've spoken with support and sent them the relevant info, I'll update this post as soon as we figure out what's going on.
Thanks!
Scott
Hi Scott,
Did support manage to assist you at all? Im having a call with them tomorrow for the same issue.
Thanks a lot!
Pne
