Forum Discussion

bhanu_bi's avatar
bhanu_bi
Frequent Visitor
2 years ago
Solved

Using Copy activity parquet file format, all columns have data type as string

I am trying to use copy activity to copy a Snowflake table into Lakehouse Files as destination. I am using Parquet file format in the destination setting. After successfully loading the files, when I...
General Question
Need Help
pipelines
  • bhanu_bi's avatar
    bhanu_bi
    2 years ago

    The issue in this case was with Snowflake Instance. We tried same workflow with Azure SQL and it worked fine. In case you are using Snowflake and facing this issue, check out the query execution in Snowflake monitoring queries and you will notice the file format is CSV, you may have to do additional tweaking to get it to parquet.

     

    If you disable the staging during this operation, pipeline doesn't work as well. That's one more thing to checkout.

bhanu_bi's avatar
bhanu_bi
Frequent Visitor
2 years ago

The issue in this case was with Snowflake Instance. We tried same workflow with Azure SQL and it worked fine. In case you are using Snowflake and facing this issue, check out the query execution in Snowflake monitoring queries and you will notice the file format is CSV, you may have to do additional tweaking to get it to parquet.

 

If you disable the staging during this operation, pipeline doesn't work as well. That's one more thing to checkout.

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi bhanu_bi ,

 

Good, thanks for your solution and feedback.

 

Best regards,

Adamk Kong