Forum Discussion

wilson_woon's avatar
wilson_woon
Regular Visitor
9 years ago
Solved

remove duplicate headers after combining multiple files

Hi all,

 

I have several csv files to be combined together into a single file for editing and analysis. Each csv file has some headers and these headers are the same. As you can imagine, these headers will be included in the file. How do I remove these redundant headers in the Query Editor? The Remove Duplicates will do the job but I feel that it would remove duplicate data as well.

 

Thanks

 

  • Eric_Zhang's avatar
    Eric_Zhang
    9 years ago

    wilson_woon

     

    As the column names are known, you can remove the header rows with Table.RemoveMatchingRows.

    Check

     

    let
    Source = Table.FromRows({{"data1","data2","data3"},{"column1","column2","column3"},{"data1","data2","data3"}},{"column1","column2","column3"}),
    RemoveHeaders = Table.RemoveMatchingRows(Source,{[column1= "column1",column2= "column2",column3= "column3"]})
in
    RemoveHeaders

2 Replies

  • Eric_Zhang's avatar
    Eric_Zhang
    Microsoft Employee
    9 years ago

    wilson_woon

     

    As the column names are known, you can remove the header rows with Table.RemoveMatchingRows.

    Check

     

    let
    Source = Table.FromRows({{"data1","data2","data3"},{"column1","column2","column3"},{"data1","data2","data3"}},{"column1","column2","column3"}),
    RemoveHeaders = Table.RemoveMatchingRows(Source,{[column1= "column1",column2= "column2",column3= "column3"]})
in
    RemoveHeaders

  • nemo2601's avatar
    nemo2601
    New Member
    3 years ago

    Hi, 

    Thanks for the solution! But wont it remove rows which are not headers with same data as well? I might be missing something, pls let me know. 

     

    Thanks !