I have a set of CSV files that require skipping the first line. I have this configured, and the preview works as expected. However, when running the pipeline, it fails with:
row number 3: found more columns than expected column count
The CSV layout is:
Row 1 > Data description label
Row 2 > Blank
Row 3 > headers
Row 4 > data
It appears that when the copy is actuall run, the skip line value is being ignored, and it fails when it gets to the headers, because it was expecting a single column based on Row 1.
So I had looked at setting the skip to 2, and the preview was incorrect, as it shows row 4 as the headers.
However, If I actually run it, it works as expected.
So the issue appears to be that the preview auto ignores/skips the blank line, but when running the pipeline, it does not.
Do you happen to have First row as header selected in your File format settings? And could you share a screenshot of your source settings and advanced source configurations?
Yes, here is the configuration, error, and source file:
Note that the preview of the data is correct:
This is the error when running the pipeline:
Update on my previous reply:
Just tested your case, uploaded a csv and tried different configurations.
It might sound counter intuitive I know, but using Skip line count = 2 breaks the preview data, it wont work as expected. But then when you actually run the pipeline, it works fine!
Here is how preview data looks
Check out the August 2023 Fabric update to learn about new features.
Become a founding member of the Data Factory Community.
Join Microsoft Reactor and learn from developers.
Want to learn more about Data Factory in Fabric? Join our webinars this month to build up your Data Factory skills!