Forum Discussion
Gen 2 dataflows extremely slow vs. Gen 1 dataflow
I'm finding Gen 2 dataflows in Fabric to be much slower than Gen 1 dataflows. Notably when pulling from an Excel file. I created a Gen 1 and Gen 2 dataflow with identical M-code in the same Fabr...
Hi. Let's consider two things here.
From one hand you should consider storage that gen2 has the possibility of a destination. That might mean converting to different formats before finishing to a storage of your capacity. The gen1 stores in black box at a microsft capacity. Gen2 also have a staging feature. If you have it turned on, it is storing two times. You can read about that here: https://blog.ladataweb.com.ar/post/738525446383288320/fabric-que-es-el-stagign-de-dataflows-gen2
On the other hand it's capacity. Gen2 only runs over capacity, it depends on the overload CUs and size of the capacity if it can run in a good shape. Gen1 can run over shared capacity of microsft that let the best of it.
I hope that helps,