This is best Fabric, Power BI, SQL and AI community event. How do we know? The last event sold out! Save €200 with code FABCMTY200.
Register nowA new Data Days event is coming soon! This time we’re going bigger than ever. Fabric, Power BI, SQL, AI and more. Don't miss out.
Fast Copy in Dataflows Gen2 is a game-changer to enhance the performance and cost-efficiency of your Dataflows Gen2. By leveraging the same optimized backend as the Copy Activity in data pipelines, Fast Copy significantly reduces data processing time and enhances cost efficiency.
Fast Copy in Dataflows Gen2 (Generally Available) enabled by default in all newly created Dataflows Gen2—making it the recommended choice for your production workloads.
In this blog post, you’ll discover how Fast Copy dramatically improves performance and efficiency when loading data from Snowflake into Microsoft Fabric.
We’ll walk through a real-world example using Dataflows Gen2 to load 180 million rows from Snowflake into a Lakehouse table. By comparing performance before and after enabling Fast Copy, you’ll see the substantial impact it can make.
Configurations Steps to reproduce this scenario:
Tip: You can use any similar dataset, such as the NYC Taxi dataset.
Boost_Performance_with_Fast_Copy_in_Dataflows_Gen2_for_Snowflake
Performance Result
The Dataflow Gen2 refresh took approximately 42 minutes to ingest 180M rows.
Boost_Performance_with_Fast_Copy_in_Dataflows_Gen2_for_Snowflake
Configurations Steps to reproduce this scenario:
Performance Result
With Fast Copy enabled, the same scenario completes in about 5 minutes.
Boost_Performance_with_Fast_Copy_in_Dataflows_Gen2_for_Snowflake
This table compares the performance of the two scenarios:
Boost_Performance_with_Fast_Copy_in_Dataflows_Gen2_for_Snowflake
With Fast Copy enabled in Dataflows Gen2, data processing times are significantly reduced. In this example, loading 180 million rows from Snowflake into a Lakehouse in Microsoft Fabric resulted in an 8X improvement in performance.
Give it a try yourself and experience the performance boost firsthand!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.