Deployment Pipeline Error

[km267]

In order to ensure that my pipelines when deployed from dev to test point to the data sources in the test workspace, I decided to create a lookup for my data sources and sinks. The data source parameters (database or lakehouse Id, workspace Id, connection Id) are then passed as parameters into the pipeline. 

In JSON view, the activity properties ia written thus:

"typeProperties": {

                            "workspaceId": "@pipeline().DataFactory",

                            "artifactId": "@variables('Database_ID')"

When I attempt to deploy the pipeline to test, it fails with 

• Backend ErrorImport failure: RequestValidationFailed. 'WorkspaceId' cannot be null.

Has anyone else encountered this issue? What is the work around?

[v-karpurapud · Accepted solution]

Hi @km267 

I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, please Accept it as a solution and give it a 'Kudos' so others can find it easily.

Thank you.

[v-karpurapud]

Hi @km267 

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.

Thank you.

[km267]

I did not write this  expression @pipeline().DataFactory on purpose. I simply selected the Workspace ID found in system variable and it returned this - @pipeline().DataFactory. If this doesn't belong in fabric it may need to be removed.

Are you asking that I hardcode the workspace id as a parameter? If so, does this not then mean that I have to hardcode it across all 3 environments? This defeats the purpose of the deployment. It still didn't work when I tried it. I still got same error: Import failure: "RequestValidationFailed. 'WorkspaceId' cannot be null".

 

Also, could you please point me to a guideline for configuring deployment rules for data pipelines?

[v-karpurapud]

Hi @km267 

The error "RequestValidationFailed. 'WorkspaceId' cannot be null" occurs because the @pipeline().DataFactory expression automatically inserted when selecting Workspace ID in Microsoft Fabric is an Azure Data Factory system variable that is not supported in Fabric. Consequently, it resolves to null during deployment, causing the failure.

Moreover, Microsoft Fabric currently does not support deployment pipeline parameter rules for data pipeline connection properties such as workspaceId and artifactId. This limitation likely explains why your parameter-based deployment approach also failed, despite following the correct pattern for defining and referencing parameters.

As a workaround, you can use a Web activity at the beginning of your pipeline to dynamically retrieve the required connection metadata, such as workspaceId and artifactId (e.g., a lakehouse or warehouse ID), via the Microsoft Fabric REST API, specifically the “List items” endpoint under Lakehouses or Warehouses. Store the retrieved values in pipeline variables using a Set Variable activity. 

To call the API, configure the Web activity with OAuth 2.0 authentication using an Azure AD app registration that has the necessary Microsoft Graph or Fabric-specific permissions. Parse the JSON response using an expression or data flow, extract the required ID(s), and assign them to variables.

For more detailed understanding, please refer

 Dynamic Warehouse & Lakehouse Connections in Microsoft Fabric Data Pipelines | Under the kover of bu...


If this post helps, kindly mark it as Accepted Solution.

Thank You!

[v-karpurapud]

Hi @km267 

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

[v-karpurapud]

Hi @km267 

Welcome to the Microsoft Fabric Forum.
 
The error 'WorkspaceId' cannot be null occurs because the expression @pipeline().DataFactory is not valid in Microsoft Fabric. This expression is specific to Azure Data Factory and is not supported in Fabric pipelines. To resolve this issue, define the necessary values such as workspaceId and artifactId as pipeline parameters, and reference them using @pipeline().parameters.workspaceId and @pipeline().parameters.artifactId in your activity JSON. Then, use the deployment pipeline interface to create parameter rules that map these parameters to the correct values for each environment, including development, test, and production. This method ensures that the pipeline receives the correct environment-specific values during deployment.

If this information helps resolve your issue, kindly consider marking this response as the Accepted Solution, as it may assist other community members facing similar challenges.
 
Thank You!