Using Copy activity parquet file format, all columns have data type as string

[bhanu_bi]

I am trying to use copy activity to copy a Snowflake table into Lakehouse Files as destination. I am using Parquet file format in the destination setting. After successfully loading the files, when I am using notebook to create table based on that files, I am getting all the column datatype as string.

I have updated the mapping in Copy activity however still the same.

As seen O_ORDERKEY is showing as string instead of number.

Is this an expected behavior?

Is there something I am missing?

Can you suggest how to pick the data type which has already been defined in mapping for copy activity.

[bhanu_bi]

Hi @Anonymous Thank you for the tip. We enabled the check-box, however still the same behavior is persisting.

We are copying from Snowflake SQL to Lakehouse Files in parquet format.

In your screenshot - Processed Table is inside Lakehouse Tables location.

Any processing that needs to be done.

Thank you.

[bhanu_bi · Accepted solution]

The issue in this case was with Snowflake Instance. We tried same workflow with Azure SQL and it worked fine. In case you are using Snowflake and facing this issue, check out the query execution in Snowflake monitoring queries and you will notice the file format is CSV, you may have to do additional tweaking to get it to parquet.

If you disable the staging during this operation, pipeline doesn't work as well. That's one more thing to checkout.

[Anonymous]

Hi @bhanu_bi ,

Good, thanks for your solution and feedback.

Best regards,

Adamk Kong

[Anonymous]

Hi @bhanu_bi ,

When you change the data type during the mapping process, you need to check the box in front of the specific field that needs to be changed. Otherwise the result of copy data will not take effect.

successfully changed:

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.