Forum Discussion
Gen2 DataFlows
I think there is a maximum of 50 tables per Data Flow. If you have that many and are worried about performance, I suggest you break up the transformations into a second step, like in a stored procedure that moves data from the lakehouse to a warehouse. This frees up the Data Flow to be a straight-foward 'move data as fast as you can' operation.
Look into the 'medallion architecture' philosphy.
Super UserHello Anonymous ,
Single dataflow has a limit of 50 tables.
Reference: Power Query Online Limits
Best Practices for Managing Multiple Tables in Dataflows
If you have a large number of tables, I recommend implementing staging tables for optimal performance.
- If your data source is SQL and some steps support query folding, you can create an initial dataflow to take advantage of folding.
- After that, you can link this dataflow to a new one for additional transformations that do not support query folding.
Note: Fully query-folded operations significantly improve performance.
More details on query folding:
Boosting Power BI Performance with Query Folding Techniques
Fast Copy in Dataflow Gen2
The Fast Copy feature in Gen2 Dataflows enhances performance. However, please note that Cloud Gateway does not support Fast Copy.
More details on Fast Copy:
Accelerating Data Loads with Fast Copy in Fabric Dataflows
Kind Regards,
Gรถkberk Uzuntaล
๐ If this post helps, then please consider Accepting it as a solution and giving Kudos โ it helps other members find answers faster!
๐ Stay Connected:
๐ Medium |
๐บ YouTube |
๐ผ LinkedIn |
๐ท Instagram |
๐ฆ X |
๐ฝ Reddit |
๐ Website |
๐ต TikTok |