Forum Discussion
Dataflow runs for hours
Hello amolt
Thanks for using the Fabric community.
As I understand the current inquiry, the question is why Dataflow Gen2 is taking more time than usual. Please correct me if I'm mistaken.
You mentioned that it's 'consuming all the resources of my capacity,' which seems unusual and suggests that there may be other activities occurring concurrently, such as another scheduled data flow. If you're uncertain, you could attempt to reschedule the Dataflow during off-hours to see if this adjustment results in fewer failures.
Thanks
HImanshu
Hello HimanshuS-msft ,
Thank you for your reply.
No, I don't have any other treatment in parallel. During this last execution, the dataflow lasted more than 5h against a few minutes usually.
Thanks
Hi DataPne , Joshrodgers123 ,
Do you use a pipeline to execute your dataflows ?
In my case, I noticed that my problems came when I used a pipeline to schedule the execution of the gen2 dataflow.Hi amolt ,
The issue seems to present itself even just when executing the pipelines manually from the UI.
It will maybe run successfully 1/10 times.
Have you managed to find any work arounds on your side?
