Fabric COPY UserErrorInvalidCopyBehaviorBlobNameNotAllowedWithPreserveOrFlattenHierarchy gzip

[FabricNZ]

Hi All,

What should be a simple Gzip copy (single csv file inside) to Fabric, is not.  I have tried all combination of the copy option. The goal is to move the GZIP from AWS S3 to Lakehouse and land it as a CSV.  I have tried compression at source and none on the destination. All manner of preserve and flatten hierachies.  Recursive and non.  

S3 connection is fine and I am pointing at a single file which I can download no problem outside of Fabric.

I have tried destination file names with .gz, csv and no extension.  Have tried GZ to GZ compression. GZ -> no compression.

For the copy behaviour I only see preserve, flatten & custom.

ErrorCode=UserErrorInvalidCopyBehaviorBlobNameNotAllowedWithPreserveOrFlattenHierarchy,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Cannot adopt copy behavior PreserveHierarchy when copying from folder to a single file.,Source=Microsoft.DataTransfer.ClientLibrary,'

I would love a little help please.  Thank you.

[FabricNZ · Accepted solution]

Okay, I solved my own problem and thought I'd close the loop on this to save people some time if they hit the same thing. When I copied GZ from S3 to Lakehouse, it failed. I was using binary and fastest compression type. By changing the compression on both the source and the destination it worked. Go figure!  Why does a binary copy care about the contents? "cp" for example doesn't.  Anyways, thought that may assist. Also for decompressing a gz file with a csv, choose delimited text and then gz from there.  Don't waste your time thinking you should start from the gz and crack that open. It is all very counter intutive, but hopefully that saves people a few days, some head scratching and some hair.

[v-saisrao-msft]

Hi @FabricNZ 
Thank you for your valuable feedback.

[FabricNZ]

Hey there, yes there are three options. Add Dynamic Content is one of them. There is also a custom expression next to the filename component.  In this expression I have tried a few things such as file.csv or a filename made up from concat with a csv extension.

As for the other options, I have tried Flatten and get the same error.  I had also tried Preserve and get the same error.  I have tried the binary option to be both with GZ compression and without it.  I am copying from a single file, i.e. a single GZ on S3, but it seems to think it is a folder.  When I put a custom expression in the copy behaviour I get copy behaviour is not specified.  I have to be missing a simple trick 😞  But, this feels like it is one of those things that should just work. Thank you for your prompt response.

[v-saisrao-msft]

Hi @FabricNZ 
Thank you for reaching out to Microsoft Forum community.

Could you please confirm if, as mentioned in the query, the copy behavior has three options? Have you tried using the custom copy behavior?

This mode is designed for defining specific file operations and is suitable for managing individual files rather than entire folders.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you.