Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago
Solved

Getting data load error from azure blob to warehouse using wildcard file path copy

Hello,

I am using a data pipeline to copy the csv files from azure blob storage to Fabric warehouse using wildcard file path. I created the structure already in warehouse tables as how it will expect. But when am loading them using pipeline it is giving me the error which i am not able to reproduce it is saying modified date column is not matching with external data type.

But the thing here is same file is loading successfully using filepath scenario not using wildcard file path.

Success scenario - loading the file using filepath and data is loading successfully to warehouse table.

 

 

 

 

Failure scenario - Here am using wildcard file path and data load is failing to target warehouse table.

 

 

Error Message - Message=SQL DW Copy Command operation failed with error 'Column 'ModifiedDate' of type 'DATETIME2' is not compatible with external data type 'Parquet physical type: BYTE_ARRAY, logical type: UTF8', please try with 'VARCHAR(8000)'

 

Do anyone know why this is happening its working with single file path but using wildcard filepath its failing. ALso i checked the target warehouse table column type it is Datetime2(3). It is saying date field is compatible but varchar(8000) is a string ryt how can it work why it is saying its not compatible any idea. Did anyone faced this issue ?

2 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Thank you so much i went through the documentation shared wildcard characters should be passed correctly to fetch or load the data into target system. I retested them with required folders and specified characters it worked for me.