Forum Discussion

MashfiqueFahim's avatar
MashfiqueFahim
New Member
4 months ago

CopyJob → Snowflake: Database name is hard‑coded in activity JSON and cannot be parameterised

When using CopyJob in Microsoft Fabric to load data into Snowflake, the selected Database becomes permanently embedded inside the CopyJob’s internal JSON definition. Unlike the connection (server, user, warehouse), the Database is not part of the connection object and is not exposed to dynamic content or environment variables.

As a result, when deploying pipelines from Dev → SIT → UAT using Deployment Pipelines, the CopyJob in SIT/UAT continues to write to the Dev Snowflake database, even though the connection has been correctly parameterised and switched to SIT/UAT.

The CopyJob JSON shows the database name hard‑coded, and this JSON is not editable in the UI or via dynamic content.

This prevents environment‑agnostic pipelines and breaks standard Dev→Test→Prod deployment patterns.

Detailed Problem Description
1. Fabric Snowflake connections do not include a Database field
The Snowflake connector only supports:

Server (account URL)

Username

Password

Warehouse

There is no field for Database or Schema.

2. CopyJob stores the Database inside the activity JSON, not the connection
When configuring a CopyJob in Dev:

The user selects a Snowflake database (e.g., DATA_LAKE_DEV)

Fabric stores this value inside the CopyJob JSON

This value is not parameterisable

This value is not editable after deployment

Dynamic content is not supported for this field

3. The hard‑coded JSON looks like this
Inside the CopyJob definition, the destination section contains:
"destination": {
"type": "SnowflakeTable",
"connectionSettings": {
"type": "Snowflake",
"typeProperties": {
"database": "DATA_LAKE_DEV"
}
}
}
This JSON is not exposed in the UI, not editable, and not overridable via variable libraries.

4. Deployment Pipelines cannot adjust the Snowflake database
After deploying to SIT:

The connection correctly switches to SIT

But the CopyJob still contains "database": "DATA_LAKE_DEV"

The UI does not allow changing the database

Dynamic content is not supported

Incremental CopyJob mode does not support Query mode (so no workaround)

5. Result: SIT and UAT pipelines write into the Dev database
This breaks:

Environment isolation

Data governance

CI/CD deployment patterns

Automated promotion workflows

Impact
CopyJobs cannot be made environment‑aware.

Pipelines deployed to SIT/UAT continue writing into Dev Snowflake databases.

Incremental loads cannot be parameterised.

Deployment Pipelines lose their value for Snowflake workloads.

Teams must manually recreate CopyJobs in each environment, which is error‑prone and not scalable.


Requested Enhancements
1. Add Database and Schema fields to the Snowflake connection object
This would align Fabric with ADF and allow environment‑specific connections.

2. Allow Database and Schema to be parameterised in CopyJob
Expose these fields to dynamic content so variable libraries can control them.

3. Allow editing the Database/Schema in CopyJob after deployment
This would allow SIT/UAT to rebind the target database without recreating the activity.

 

5 Replies

  • v-sathmakuri's avatar
    v-sathmakuri
    Community Support

    Hi  MashfiqueFahim , 

    Thank you for reaching out to Microsoft Fabric Community.

     

    The behavior you are encountering is due to the current design of Microsoft Fabric, where the Snowflake database and schema are defined at the CopyJob activity level and stored as static properties within the activity definition. These values are not part of the connection object and are not exposed for parameterization or modification through Deployment Pipelines. As a result, during Dev -> SIT ->UAT promotion, only the connection is updated, while the database embedded in the CopyJob remains unchanged, leading to data being written to the original (Dev) database.

     

    As recommonded by  tayloramy , please submit your requirement in the ideas forum, to implement it in the future releases.  


    As an alternative to meet the requirements, it's recommended to use a Data Pipeline with Copy Activity instead of CopyJob, as this approach supports parameterization and dynamic content. With this setup, you can define the Snowflake database and schema as pipeline parameters and manage them for each environment through Deployment Pipeline rules, ensuring environment isolation and CI/CD compatibility. If you choose to use CopyJob, you will need to create and maintain separate CopyJobs for each environment (Dev, SIT, UAT) and manually set the correct Snowflake database in each, since the database value can't be changed dynamically after deployment.

     

    Thanks!!

  • Lozovskyi's avatar
    Lozovskyi
    Kudo Collector

    Hello MashfiqueFahim 

    thank you for raising your question.

    Could you please check and confirm it's still not possible to have database name as parametrized value?

    I've just checked in one of the solution, and it's supported.

    SOmetimes, you can simply try pressing Alt+Shift+D to check if this field supported to be parametrized.

    If there is no luck, you can also try creating new connection and and copy activity to ensure it's up to date.

     

    Hope it helps.

     

    BR, Taras