Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Be one of the first to start using Fabric Databases. View on-demand sessions with database experts and the Microsoft product team to learn just how easy it is to get started. Watch now

Reply
SamWilliams
Regular Visitor

Data pipeline Copy Data issue with CSV file and skip line

I have a set of CSV files that require skipping the first line. I have this configured, and the preview works as expected. However, when running the pipeline, it fails with:

 

row number 3: found more columns than expected column count

 

The CSV layout is:

 

Row 1 > Data description label

Row 2 > Blank

Row 3 > headers

Row 4 > data

 

It appears that when the copy is actuall run, the skip line value is being ignored, and it fails when it gets to the headers, because it was expecting a single column based on Row 1.

7 REPLIES 7
SamWilliams
Regular Visitor

So I had looked at setting the skip to 2, and the preview was incorrect, as it shows row 4 as the headers.

 

However, If I actually run it, it works as expected.

 

So the issue appears to be that the preview auto ignores/skips the blank line, but when running the pipeline, it does not.

 

n0elleli
Community Admin
Community Admin

Hi there!

Do you happen to have First row as header selected in your File format settings? And could you share a screenshot of your source settings and advanced source configurations?

Yes, here is the configuration, error, and source file:

 

SamWilliams_0-1685709178830.png

 

SamWilliams_1-1685709214478.png

 

Note that the preview of the data is correct:

SamWilliams_2-1685709252606.png

This is the error when running the pipeline:

 

Error
 
Troubleshoot activity failures
{ "errorCode": "2200", "message": "ErrorCode=DelimitedTextMoreColumnsThanDefined,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error found when processing 'Csv/Tsv Format Text' source 'DataflowTest.csv' with row number 3: found more columns than expected column count 1.,Source=Microsoft.DataTransfer.Common,'", "failureType": "UserError", "target": "Copy data1", "details": [] }
 
Source CSV File (5 rows total: label, blank, headers, data):
 
This is a data file header

Column A,Column B,Column C
Row 4A,Row 4B,Row 4C
Row 5A,Row 5B,Row 5C
 
 

Update on my previous reply: 

 

Just tested your case, uploaded a csv and tried different configurations.

 

It might sound counter intuitive I know, but using Skip line count = 2 breaks the preview data, it wont work as expected. But then when you actually run the pipeline, it works fine!

 

MartinZurita_0-1685719784868.png

MartinZurita_1-1685719837384.png

 

 

Here is how preview data looks

MartinZurita_2-1685719876748.png

 

Thanks @MartinZurita. @SamWilliams can you try whether it works for you? While we will log this bug for inconsitent between data preview and run.

Yes, setting skip to 2 works for the actual run, but not preview.

Sam

Have you tried with skip = 2?

 

 

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount!

November Carousel

Fabric Community Update - November 2024

Find out what's new and trending in the Fabric Community.

Dec Fabric Community Survey

We want your feedback!

Your insights matter. That’s why we created a quick survey to learn about your experience finding answers to technical questions.