Cookistador's avatar
Cookistador
Icon for Super User rankSuper User
9 months ago
Status:
New

[Fabric Copy Job] Unable to parameterize SQL Database name for Deployment Pipelines (CI/CD)

I spotted an issue with the Fabric Copy Job item regarding the parameterization of SQL Server sources within Deployment Pipelines.

 

Scenario: I am using a Copy Job to ingest data from an on-premises SQL Server to a Fabric Lakehouse. My environment utilizes Deployment Pipelines to promote content from Dev to UAT/Prod. I am attempting to use Variable Libraries to handle connection differences across these environments.

 

The Issue: While I can successfully bind the SQL Server Connection object to a Library Variable, the Database Name field in the source settings does not support parameterization or variable binding. It remains hardcoded.

Consequently, when I deploy the Copy Job from the Development workspace to UAT/Prod, the job continues to target the hardcoded Development database name (e.g., fabnavtest1), which is incorrect for the target environment.

 

Troubleshooting Steps Taken:

  1. Verified that the Connection ID is successfully parameterized using the Variable Library VL_FDL_Config.

  2. Attempted to locate the underlying JSON definition via OneLake Explorer (similar to the workaround available for Dataflow Gen 2 destination issues) to manually inject a parameter, but the Copy Job folder appears empty/inaccessible.

    Technical Evidence: Below is the JSON definition of the source. You can see that connectionVariableLibrary is set correctly, but the database property is a hardcoded string with no option to bind it to a variable.

    JSON
     
    {
      "properties": {
        "jobMode": "Batch",
        "source": {
          "type": "SqlServerTable",
          "connectionSettings": {
            "type": "SqlServer",
            "typeProperties": {
              "database": "fabnavtest1" 
            },
            "externalReferences": {
              "connectionVariableLibrary": {
                "variableName": "SQL_CONN_ID",
                "libraryName": "VL_FDL_Config"
              }
            }
          }
        }
      }
    }

    Impact: This prevents the successful use of Deployment Pipelines for Copy Jobs using SQL Server sources, as we cannot dynamically change the database target per environment.

     

    Thank you for your help

1 Comment

  • Just witnessed the same issue. The connection GUID is correctly getting the active set variable value, but the copy job JSON "database" still has the original connections database name.