topic Lakehouse source not supported by fast copy? in Dataflow

Lakehouse source not supported by fast copy?

frithjof_v — Sat, 23 Nov 2024 15:37:24 GMT

let Source = Lakehouse.Contents(null){[workspaceId = "..."]}[Data]{[lakehouseId = "..."]}[Data], #"Navigation 1" = Source{[Id = "Fact_Order", ItemKind = "Table"]}[Data] in #"Navigation 1"

Why does the fast copy indicator say that This source is not supported by fast copy?

Isn't Lakehouse source supported for fast copy?

Re: Lakehouse source not supported by fast copy?

frithjof_v — Sat, 23 Nov 2024 15:49:28 GMT

The dataflow actually seems to run successfully, even if I have checked the Require fast copy

Does it mean that the Fast copy indicator, which told me that this source is not supported by Fast copy, is wrong?

If so, that indicator should be fixed 🙂

Re: Lakehouse source not supported by fast copy?

uzuntasgokberk — Sun, 24 Nov 2024 14:53:07 GMT

Hello frithjof_v ,

İf you would like to use fast copy there is requirements. İf you'd like know more about fast copy below the link you can follow:
My medium post what I've published recently: h target=_blank target=_blank target=_blankttps://medium.com/microsoft-power-bi/accelerating-data-loads-fast-copy-in-fabric-dataflow-67d743506a22
Microsoft learn fast copy:https://learn.microsoft.com/en-us/fabric/data-factory/dataflows-gen2-fast-copy

Requirements for Enabling Fast Copy

Fast Copy is currently supported for Dataflow Gen2 connectors including ADLS Gen2, Blob Storage, Azure SQL DB, Lakehouse, PostgreSQL, On-Premises SQL Server, Warehouse, Oracle, and Snowflake.
For file data, files must be in .csv or Parquet format, with a minimum size of 100 MB, and stored in Azure Data Lake Storage (ADLS) Gen2 or Blob storage. For databases, such as Azure SQL DB and PostgreSQL, the data source must contain at least 5 million rows.
Data Gateway must be set to On-Premises(version 3000.214.2 or later), Cloud and VNet gateway support is not available for the Fast Copy process.
Currently, Fast Copy supports only in Fabric Dataflow. For more information about Fabric Dataflow, you can refer to U target=_blank target=_blank target=_blanknderstanding DataFlow Gen2 in Microsoft Fabric And Comparison with DataFlow Gen1.
Any transformation(applied steps in Power Query) included in the native query is supported such as Select columns, Rename a column etc.

Best Regards,
Gökberk Uzuntaş
LinkedIn: https://www.linkedin.com/in/g%C3%B6kberk-uzunta%C5%9F-b43906198/
Medium: https://medium.com/@uzuntasgokberk

İf this post helps, then please consider Accept it as solution and kudos to help the other members find it more quickly.

Re: Lakehouse source not supported by fast copy?

frithjof_v — Sun, 24 Nov 2024 20:08:00 GMT

I am fulfilling all of those requirements, still the indicator tells that fast copy doesn't work.

But when I run the dataflow, the fast copy works.

So I think there is a bug with the indicator in the applied steps.

Re: Lakehouse source not supported by fast copy?

uzuntasgokberk — Sun, 24 Nov 2024 20:44:07 GMT

Hello frithjof_v ,

To make sure your df works fast copy please follow below the image:

You df needs to be work engine that is CopyActivity. Otherwise it doesn't work with FastCopy that means also you need to follow every step in requirements. İf you see CopyEngine, you can raise a ticket about your issue to Microsoft.

Best Regards,
Gökberk Uzuntaş

LinkedIn: https://www.linkedin.com/in/g%C3%B6kberk-uzunta%C5%9F-b43906198/

Medium: https://medium.com/@uzuntasgokberk

İf this post helps, then please consider Accept it as solution and kudos to help the other members find it more quickly.

Re: Lakehouse source not supported by fast copy?

frithjof_v — Sun, 24 Nov 2024 21:29:10 GMT

Yes, it shows as CopyActivity in the refresh log:

But the applied steps indicator says that Fast Copy is not supported for that data source.

So there seems to be a bug with the Fast Copy indicator in the Applied steps:

The source is Fabric Lakehouse.