topic Re: Fabric Pipeline - Copy Data - DateTime2 column mapping issue in Pipelines

Fabric Pipeline - Copy Data - DateTime2 column mapping issue

smoqt — Mon, 29 Apr 2024 16:39:15 GMT

I am trying to load a Fabric Data Warehouse table from csv using the Copy Data activity in a pipeline.

I have a datetime2 column in my table called __timestamp that is failing.

The error:

'Column '__timestamp' of type 'DATETIME2' is not compatible with external data type 'Parquet physical type: BYTE_ARRAY, logical type: UTF8', please try with 'VARCHAR(8000)'

Here is the column definition.

Here is the __timestamp column from the file.

Here is the inferred mapping.

File format settings (first row as header is checked):

This works fine when loading the same file to the same table using ADF. I'd prefer not to change the datatype of the underlying column to VARCHAR(8000) as suggested. Can you help me understand this?

Re: Fabric Pipeline - Copy Data - DateTime2 column mapping issue

Anonymous — Tue, 30 Apr 2024 15:41:39 GMT

Hi smoqt
Thanks for using Fabric Community.
At this time, we are reaching out to the internal team to get some help on this. We will update you once we hear back from them.
Thanks

Re: Fabric Pipeline - Copy Data - DateTime2 column mapping issue

Anonymous — Tue, 07 May 2024 05:11:13 GMT

Hi smoqt
Apologies for the delay in response.
The internal team wants you to create a support ticket here to understand the issue clearly.
I will request you to please go ahead with Microsoft support on this. 
https://support.fabric.microsoft.com/en-US/support/
After creating the ticket please provide the details here. This would be helpful for internal tracking.
Thanks

Re: Fabric Pipeline - Copy Data - DateTime2 column mapping issue

Anonymous — Wed, 08 May 2024 17:15:03 GMT

Hi smoqt
We haven’t heard from you on the last response and was just checking back to see if you got a chance to create a support ticket. Otherwise, will respond back with the more details and we will try to help.
Thanks

Re: Fabric Pipeline - Copy Data - DateTime2 column mapping issue

smoqt — Wed, 08 May 2024 21:20:08 GMT

Hi, Anonymous. A ticket was created through our partner. Thank you.

Re: Fabric Pipeline - Copy Data - DateTime2 column mapping issue

Anonymous — Thu, 09 May 2024 01:47:22 GMT

Hi smoqt

Can you please provide the ticket details?

Thanks