Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
1 year ago
Solved

Design Question

What are the best practices for handling large file transfers in Azure Data Factory, and how can I optimize performance for big data workflows?
  • frithjof_v's avatar
    1 year ago

    If you are referring to Fabric Data Factory:

     

    Try using Data Pipeline Copy Activity, or Dataflow Gen2 Fast Copy.

     

    (If you are referring to Azure Data Factory: that's another forum.)

     

    You could potentially look into Notebooks as well.

     

    What kind of data source are you going to read from?

  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi Anonymous ,

    If it's in Fabric Data Factory, you can test as frithjof_v said.

     

    For your second question, to optimize performance for big data workflows in Azure Data Factory.

    I think you can check out this official document below:

    Mapping data flow performance and tuning guide - Azure Data Factory & Azure Synapse | Microsoft Learn

     

     

    Best Regards

    Yilong Zhou

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.