Error using a get metadata activity with a parameterized lakehouse source

[pmscorca]

Hi,

I've created a data pipeline with a get metadata activity.

I've added a pipeline parameter in order to pass the name of the source lakehouse (Bronze).

In the files of the Bronze lakehouse I've loaded some csv files.

I need to consider these csv files by the get metadata activity, where I've specified 

@pipeline().parameters.lakehouse_name as a connection, Lakehouse as a connection type while the workspace ID is filled automatically.

When I execute the data pipeline I obtain this error:

Field 'exists' failed with error: 'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Lakehouse operation failed for: Operation returned an invalid status code 'BadRequest'. Workspace: '388e9804-xxxx-wwww-yyyy-e618ca42198b'. Path: 'Bronze/Files'. ErrorCode: 'BadRequest'. Message: 'BadRequest'. TimeStamp: 'Sun, 30 Jun 2024 17:00:07 GMT'..,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.Azure.Storage.Data.Models.ErrorSchemaException,Message=Operation returned an invalid status code 'BadRequest',Source=Microsoft.DataTransfer.ClientLibrary,'.

The data pipeline runs successfully when I specify the lakehouse name for the connection, without parameterizing it.

Any helps to me about this issue, please? Thanks

[Anonymous · Accepted solution]

Hi @pmscorca ,

Based on the error message you've provided, it seems like the pipeline is having trouble interpreting the parameterized connection string, leading to a 'BadRequest' error when attempting to access the specified lakehouse.

So I think first you need to make sure that the parameters exist and are passed correctly at runtime when using @pipeline().parameters.lakehouse_name. The error message mentions a specific path "Bronze/Files". Check that the path is formatted correctly and is accessible.

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.