Forum Discussion

pmscorca's avatar
pmscorca
Kudo Kingpin
2 years ago
Solved

Error using a get metadata activity with a parameterized lakehouse source

Hi,

I've created a data pipeline with a get metadata activity.

I've added a pipeline parameter in order to pass the name of the source lakehouse (Bronze).

In the files of the Bronze lakehouse I've loaded some csv files.

I need to consider these csv files by the get metadata activity, where I've specified 

@pipeline().parameters.lakehouse_name as a connection, Lakehouse as a connection type while the workspace ID is filled automatically.
When I execute the data pipeline I obtain this error:
Field 'exists' failed with error: 'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Lakehouse operation failed for: Operation returned an invalid status code 'BadRequest'. Workspace: '388e9804-xxxx-wwww-yyyy-e618ca42198b'. Path: 'Bronze/Files'. ErrorCode: 'BadRequest'. Message: 'BadRequest'. TimeStamp: 'Sun, 30 Jun 2024 17:00:07 GMT'..,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.Azure.Storage.Data.Models.ErrorSchemaException,Message=Operation returned an invalid status code 'BadRequest',Source=Microsoft.DataTransfer.ClientLibrary,'.
 
The data pipeline runs successfully when I specify the lakehouse name for the connection, without parameterizing it.
Any helps to me about this issue, please? Thanks
pipelines
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi pmscorca ,

    Based on the error message you've provided, it seems like the pipeline is having trouble interpreting the parameterized connection string, leading to a 'BadRequest' error when attempting to access the specified lakehouse.

    So I think first you need to make sure that the parameters exist and are passed correctly at runtime when using @pipeline().parameters.lakehouse_name. The error message mentions a specific path "Bronze/Files". Check that the path is formatted correctly and is accessible.

     

     

     

    Best Regards

    Yilong Zhou

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

1 Reply

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi pmscorca ,

    Based on the error message you've provided, it seems like the pipeline is having trouble interpreting the parameterized connection string, leading to a 'BadRequest' error when attempting to access the specified lakehouse.

    So I think first you need to make sure that the parameters exist and are passed correctly at runtime when using @pipeline().parameters.lakehouse_name. The error message mentions a specific path "Bronze/Files". Check that the path is formatted correctly and is accessible.

     

     

     

    Best Regards

    Yilong Zhou

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.