topic Re: Copy SQL Table to External Lakehouse File in Pipelines

Copy SQL Table to External Lakehouse File

Anonymous — Wed, 07 Aug 2024 21:34:53 GMT

I am trying to copy a table from SQL Server to a Parquet file in one Workspace to another External Workspace & Lakehouse. I am the owner of both and I can browse the destination file. The Destination side of the Copy Data is specified with variables and it browses fine.

Per instructions, I use the WorkspaceID and LakehouseIDs as GUIDs to define the connection.

If I browse it, it copies up fine, but on the copy I get the following error. Seems to

not like the GUID as part of the actual destination


ErrorCode=LakehouseOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,
Message=Lakehouse operation failed for: 'filesystem' does not match expected pattern 
'^[$a-z0-9](?!.*--)[-a-z0-9]{1,61}[a-z0-9]$'.. Workspace: '"26afc951-5c69-4f44-9652-5bd526c93b9c"'. 
Path: '"2a36c340-3ca8-475b-abd2-dbc799dccd94"/Files/erds/some_data.parquet'..,
Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.Rest.ValidationException,Message='filesystem' does not match expected pattern '^[$a-z0-9](?!.*--)[-a-z0-9]{1,61}[a-z0-9]$'.,Source=Microsoft.DataTransfer.ClientLibrary,'

Re: Copy SQL Table to External Lakehouse File

Anonymous — Thu, 08 Aug 2024 01:21:17 GMT

Hi Anonymous

According to the error message you provided, the error you encountered was due to a mismatch between the format of the file system name and the expected pattern.

This pattern recommends that file system names should contain only lowercase letters, numbers, and hyphens, and should not begin or end with a hyphen, nor should they contain consecutive hyphens.

You can check the file system name to ensure that the file system name in the target path matches the specified schema.

If you use GUids as part of the file system name, consider converting them to a format that matches the schema.

Regards,

Nono Chen

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Re: Copy SQL Table to External Lakehouse File

Anonymous — Thu, 08 Aug 2024 12:24:26 GMT

This is a valid pattern for a file system, according to the regular expression. The issue may be the quotes around it. The quotes are required for the browse to work, but may break the destination path.

Re: Copy SQL Table to External Lakehouse File

Anonymous — Tue, 13 Aug 2024 12:27:29 GMT

This is still an issue, using a GUID as the Workspace and LakehouseID - where I have ownership and permissions - should work for destination of Copy Data, but does not as it is not accepted for destination. This is a bug in my opinion.