Forum Discussion
Post ProdigyHow to handle quotes within a string in Copy Activity?
Greetings, community. I have a CSV file I am trying to load into a Lakehouse using a Copy Activity and am having some trouble related to characters in the file. Here is a fake data sample:
"ColumnA","ColumnB"
"1","Alpha"
"2","Alphabet
"Soup""
"3","Beta"
I believe the error is being caused by the quotes around "Soup" as they aren't escaped in the file from my data source. The final output I want to see in a lakehouse table is:
Does anyone know of a way I can configure my Copy activity's settings to solve for this?
Hi arpost ,
As I understand, without having any escape characters we cannot handle this kind of data in data pipeline. This data will be treated as bad data and make it to null. You can continue using Data flow as you were able to find a solution with the help of it.
Thank you
7 Replies
Hi arpost ,
Thanks for using Fabric Community.
I would like to know the input file format?
Is it this ?"ColumnA","ColumnB" "1","Alpha" "2,"Alphabet "Soup"" "3","Beta"
or this ?"ColumnA","ColumnB" "1","Alpha" "2,"Alphabet "Soup"" "3","Beta"
Can you please help me understand this?
arpostAnonymous, it is the second one, and this is the representation of the data in CSV format. ColumnB is a field that can contain user comments that may include everything from line breaks to quotation marks.
"ColumnA","ColumnB"
"1","Alpha"
"2","Alphabet
"Soup""
"3","Beta"Here's an example of how the data would look in a tabular format:
ColumnA ColumnB 1 Alpha 2 Alphabet
"Soup"3 Beta