Data pipeline Copy Data issue with CSV file and skip line

[SamWilliams]

I have a set of CSV files that require skipping the first line. I have this configured, and the preview works as expected. However, when running the pipeline, it fails with:

row number 3: found more columns than expected column count

The CSV layout is:

Row 1 > Data description label

Row 2 > Blank

Row 3 > headers

Row 4 > data

It appears that when the copy is actuall run, the skip line value is being ignored, and it fails when it gets to the headers, because it was expecting a single column based on Row 1.

[SamWilliams]

So I had looked at setting the skip to 2, and the preview was incorrect, as it shows row 4 as the headers.

However, If I actually run it, it works as expected.

So the issue appears to be that the preview auto ignores/skips the blank line, but when running the pipeline, it does not.

[n0elleli]

Hi there!

Do you happen to have First row as header selected in your File format settings? And could you share a screenshot of your source settings and advanced source configurations?

[SamWilliams]

Yes, here is the configuration, error, and source file:

Note that the preview of the data is correct:

This is the error when running the pipeline:

Error

Troubleshoot activity failures

{ "errorCode": "2200", "message": "ErrorCode=DelimitedTextMoreColumnsThanDefined,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Error found when processing 'Csv/Tsv Format Text' source 'DataflowTest.csv' with row number 3: found more columns than expected column count 1.,Source=Microsoft.DataTransfer.Common,'", "failureType": "UserError", "target": "Copy data1", "details": [] }

Source CSV File (5 rows total: label, blank, headers, data):

This is a data file header

Column A,Column B,Column C

Row 4A,Row 4B,Row 4C

Row 5A,Row 5B,Row 5C

[MartinZurita]

Update on my previous reply: 

Just tested your case, uploaded a csv and tried different configurations.

It might sound counter intuitive I know, but using Skip line count = 2 breaks the preview data, it wont work as expected. But then when you actually run the pipeline, it works fine!

Here is how preview data looks

[GraceGu]

Thanks @MartinZurita. @SamWilliams can you try whether it works for you? While we will log this bug for inconsitent between data preview and run.

[SamWilliams]

Yes, setting skip to 2 works for the actual run, but not preview.

Sam

[MartinZurita]

Have you tried with skip = 2?