topic Re: Design Question in Dataflow

Design Question

Anonymous — Tue, 12 Nov 2024 13:27:38 GMT

What are the best practices for handling large file transfers in Azure Data Factory, and how can I optimize performance for big data workflows?

frithjof_v — Wed, 13 Nov 2024 04:54:11 GMT

If you are referring to Fabric Data Factory:

Try using Data Pipeline Copy Activity, or Dataflow Gen2 Fast Copy.

(If you are referring to Azure Data Factory: that's another forum.)

You could potentially look into Notebooks as well.

What kind of data source are you going to read from?

Anonymous — Wed, 13 Nov 2024 01:50:03 GMT

Hi Anonymous ,

If it's in Fabric Data Factory, you can test as frithjof_v said.

For your second question, to optimize performance for big data workflows in Azure Data Factory.

I think you can check out this official document below:

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.