Can you skip rows in Copy activity like you can in Azure Data Factory?

[arpost]

Greetings, all. I have a scenario where I need to skip a few rows of a file in a Copy activity in order to get to the file's column headers. However, I don't see the Skip Line Count option like in Azure Data Factory. Is there something I'm overlooking? Or is the idea that we'd have to do something like pre-process the file or do it after loading?

[Anonymous · Accepted solution]

Hi @arpost ,

In Fabric Data Factory, there are several ways to solve this problem:
1. Before loading a file into Fabric Data Factory, you can use a script or other tool to preprocess the file, removing the first few lines of unneeded data. For example, use a Python script to read the file and save a new file.
2. After the file is fully loaded into the data store, you can remove unwanted lines in subsequent processing steps. For example, use the following sql statement to remove the first few rows of data:

row result:

after query:

SELECT *
FROM (
    SELECT ROW_NUMBER() OVER (ORDER BY (SELECT 1)) AS RowNum, *
    FROM HR.Employees
) AS TempTable
WHERE RowNum > 1

Best Regards,
Adamk Kong

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.