Data Job Activity

[FisayoTogun]

I have previously used Copy Data activities without any issues. Recently, the activity type was switched from Copy Data to Copy Job activities. I have no problem with this change; however, after setting everything up, I noticed that the Copy Job activity always prompts for a connection at the end—even though I already configured a connection during the activity creation process.

The reason for this requirement is unclear. The Copy Job activity uses a copy job (which already has a connection) within the same workspace, so I’m unsure why this error occurs.

I need assistance in understanding and resolving this issue.

[nielsvdc · Accepted solution]

Hi @FisayoTogun, the Copy data and Copy job activities are two different things. The Copy data activity lives within a pipeline and the Copy job lives as a stand-alone Fabric item.

The connection you need to select with a Copy job activity has nothing to do with a source or destination connection used in the Copy job. When you add a Copy Job activity to a Fabric pipeline, you’re referencing a stand-alone Copy Job object that already lives in your workspace. For the pipeline to be able to connect to the Copy job item in the workspace and execute it through the activity, you need to have a connection available for Copy jobs with an authentication that has access to the Copy job item in the workspace (preferable a service principal).

Btw, this is also the case when you want to execute other pipelines or start a semantic model refresh from a pipeline. It's a design decision in Fabric for authentication.

For your remark about that the activity type was switched to Copy job. You use both activities for different purposes. The Copy data activity is the same old activity that you are used to and can keep using as you did. For example with a foreach loop using parameters and executing dynamic queries, etc.

You can use the Copy job item for extraction and ingesting data, for example, from a SQL Server database to a Fabric lakehouse and overwrite dimension tables and do an incremental update for fact tables.

For more information about copy job see the page What is Copy job in Data Factory for Microsoft Fabric?.

Hope this helps. If so, please give a Kudos 👍 or mark as Accepted Solution ✔️.