Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started
I have a set of CSV files that require skipping the first line. I have this configured, and the preview works as expected. However, when running the pipeline, it fails with:
row number 3: found more columns than expected column count
The CSV layout is:
Row 1 > Data description label
Row 2 > Blank
Row 3 > headers
Row 4 > data
It appears that when the copy is actuall run, the skip line value is being ignored, and it fails when it gets to the headers, because it was expecting a single column based on Row 1.
So I had looked at setting the skip to 2, and the preview was incorrect, as it shows row 4 as the headers.
However, If I actually run it, it works as expected.
So the issue appears to be that the preview auto ignores/skips the blank line, but when running the pipeline, it does not.
Hi there!
Do you happen to have First row as header selected in your File format settings? And could you share a screenshot of your source settings and advanced source configurations?
Yes, here is the configuration, error, and source file:
Note that the preview of the data is correct:
This is the error when running the pipeline:
Update on my previous reply:
Just tested your case, uploaded a csv and tried different configurations.
It might sound counter intuitive I know, but using Skip line count = 2 breaks the preview data, it wont work as expected. But then when you actually run the pipeline, it works fine!
Here is how preview data looks
Thanks @MartinZurita. @SamWilliams can you try whether it works for you? While we will log this bug for inconsitent between data preview and run.
Yes, setting skip to 2 works for the actual run, but not preview.
Sam
Have you tried with skip = 2?
Check out the September 2024 Fabric update to learn about new features.
Learn from experts, get hands-on experience, and win awesome prizes.