Cannot read Parquet files from S3 to Fabric

[JoergBender]

Hey there,

I am currently exploring the opportunities of Fabric. Our main DataLake lives in AWS, therefore I want to load parquet files in one of the S3 buckets.

When I select a specific parquet file in the bucket, the Copy Activity in a Fabric pipeline returns the error " ParquetSharp does not understand the schema used". I am trying to access a valid parquet file. 

I also attached a Screenshot.

Is this a known bug? 

Best,

JB

[snaraya]

Hey @JoergBender , did you get a solution for this? i am getting same issue.

[JoergBender]

Any updates regarding the issue?

[ajarora]

Any reason why you are specifying compression type as gzip ? 

[JoergBender]

Because it is the default compression. See here.

I also tried with other compressions and it did not work.

[ajarora]

Can you please log a support ticket along with a sample file with which we can reproduce ?

[JoergBender]

This is the error from the pipeline log:

{
"errorCode": "2200",
"message": "ErrorCode=ParquetDeserializationError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=,Source=,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An unexpected error happens when reading from Parquet files.,Source=Microsoft.DataTransfer.Richfile.ParquetTransferPlugin,''Type=System.Exception,Message=ParquetSharp does not understand the schema used,Source=ParquetSharp,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An unexpected error happens when reading from Parquet files.,Source=Microsoft.DataTransfer.Richfile.ParquetTransferPlugin,''Type=System.Exception,Message=ParquetSharp does not understand the schema used,Source=ParquetSharp,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An unexpected error happens when reading from Parquet files.,Source=Microsoft.DataTransfer.Richfile.ParquetTransferPlugin,''Type=System.Exception,Message=ParquetSharp does not understand the schema used,Source=ParquetSharp,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=An unexpected error happens when reading from Parquet files.,Source=Microsoft.DataTransfer.Richfile.ParquetTransferPlugin,''Type=System.Exception,Message=ParquetSharp does not understand the schema used,Source=ParquetSharp,'",
"failureType": "UserError",
"target": "Copy Web Data Test",
"details": []
}

[GeethaT-MSFT]

Hi @JoergBender Thanks for posting your question in Microsoft Fabric Community

Currently, I am reproducing the issue and will get back to you with more details.

Regrads

Geetha

[JoergBender]

Any updates?

[GeethaT-MSFT]

Hi, @JoergBender as @ajarora mentioned can you please log a support ticket along with a sample file with which we can reproduce?

Regards

Geetha

[JoergBender]

Is there a dedicated Fabric support forum or shall I use the Azure Data Factory one? Could you share a link? Thank you.