Forum Discussion
Unable to properly merge text file using copy activity in Ms fabric.
I have two text files in Blob Storage that I’m trying to merge using a Microsoft Fabric pipeline with a copy activity. The merge works, but the first row from one of the files disappears. Has anyone else experienced this issue?
The process assumes that the files have header rows. During an append the header row of each but the first file will be discarded.
Specify that they files don't have headers, and provide your own column names.
6 Replies
- lbendlin
Super User
Did you mean to say "append" rather than "merge" ?
- lbendlin
The process assumes that the files have header rows. During an append the header row of each but the first file will be discarded.
Specify that they files don't have headers, and provide your own column names.
Hi Anonymous
Some sources and destinations might not have an option to specify whether the files have a header row. In that case, one alternative is to use the Dataflow Gen2 instead of Copy data activity. In dataflow Gen2, you can explicitly set the first row as headers or not before copying the data to the destination data storage.
Best Regards,
Jing
If this post helps, please Accept it as Solution to help other members find it. Appreciate your Kudos!Hi Anonymous
Dataflow Gen2 supports the data source Azure Data Lake Storage Gen2. Azure Data Lake Storage Gen2 connector overview
However, for the destination, it doesn't support ADLS. Dataflow Gen2 data destinations and managed settings
Best Regards,
Jing