Forum Discussion

smpa01's avatar
smpa01
Icon for Community Champion rankCommunity Champion
1 year ago

Copy Job best practice

I have to copy n tables from a on-prem SQL DB to a lakehouse with a copy job do I create n different copy jobs (similar to create n differnt dataflows) or can I create a single copy job to acco...
workspaces
Anonymous's avatar
Anonymous
Not applicable
1 year ago

Hi smpa01 ,

Thank you for reaching out to the Microsoft Fabric Community Forum.

 

As suparnababu8 mentioned, you can use a single Copy Job to transfer multiple tables from your on-premises SQL Server to a Lakehouse. When setting up the Copy Job, you can select and configure several tables in one job, which makes monitoring easier and reduces overhead. This method works best if the tables have the same refresh schedule and similar load settings.

However, if some tables need different refresh schedules or specific load strategies (such as full versus incremental loads with different watermark columns), it’s better to create separate copy jobs for those tables.

 

Hope this helps. Please reach out for further assistance.

If this post helps, then please consider to Accept as the solution to help the other members find it more quickly and a kudos would be appreciated.

 

Thank you.

 

  • smpa01's avatar
    smpa01
    Icon for Community Champion rankCommunity Champion
    1 year ago

    Copy Job is different from Data factory[Copy activity]. I was referring to the former and not the latter. Also, once I create that Copy Job for the first table, how do I go back and edit the job (adding the 2nd table for copying, adding the 3rd table for copying etc) Anonymous 

    • Anonymous's avatar
      Anonymous
      Not applicable
      1 year ago

      Hi smpa01 ,

       

      Thanks for your follow-up and for pointing out the difference between Fabric’s Copy Job and the traditional Data Factory Copy activity.

      Right now, once you create a Copy Job in Microsoft Fabric, you can’t go back and edit it to add or remove tables. If you need to include more tables, the best option is to create a new Copy Job with the updated selection.

      We understand this can be limiting, and we really appreciate your feedback. Improvements in this area are likely in future updates. If you’d like to share your thoughts directly with the product team, please consider submitting your idea through the forum link below:  Fabric Ideas - Microsoft Fabric Community

      If this post helps, then please consider to Accept as the solution to help the other members find it more quickly and a kudos would be appreciated.

       

      Thank you.