Power BI Legacy Dataflow vs Dataflow Gen2 performance

[prathyoo]

Hi,

We have an existing Power BI Legacy Dataflow with around 69 tables as follows -

We replicated this to Dataflow Gen2 in order to write all the power queries to lakehouse tables. The best part of this was that I could select all power query tables in the legaccy dataflow and use ctrl+v to copy every table and group structure over Dataflow Gen2. Recreating the dataflows just took under 10 minutes!!

Before mapping to the lakehouse, we decided to first refresh the Dataflow Gen2 to see the performance. The Power BI Legacy dataflow takes an average of 3 minutes and has been refreshing 4 times a day for the last couple of years.

Now, the dataflow gen2 seems to show the same sort of performance except for one table which is a combine of two other tables -

This table refresh in legacy dataflow takes around 15 seconds -

But currently the dataflow gen2 is stuck on this table refresh for the past one hour. Every othe table refresh is completed except for this table.

Please let me know how to improve performace for dataflow gen2 for this type of query. The source of the query is a append statement of two other tables in the dataflow.

Regards

[miguel]

Hey!

One other way to move your dataflows from Gen1 to Gen2 is using th Export template feature:

Move queries from Dataflow Gen1 to Dataflow Gen2 - Microsoft Fabric | Microsoft Learn

I don't think that we have sufficient information to provide any suggestions in terms of improving your queries, but I highly suggest reaching out to the support team by creating a support ticket so they can take a closer look at the scenario.

One workaround is to load the data to something like the Lakehouse and then do the append after the tables are in the lakehouse, but it would add an extra step.

[prathyoo]

Thanks. I am not sure whether to reach out to support as yet. The fabric workspace is still a trial capacity. The power query works fine in the legacy dataflow taking only 15 seconds and the same seems to have gone on for more than 2 hours before I had to kill the refresh.

I will raise the issue if the smae still exists after our premium capacity has moved to fabric.