Forum Discussion

ThomasRichard21's avatar
ThomasRichard21
Frequent Visitor
2 years ago

New column added in between in Source CSV, data imported from folder

Hi all,

 

My data is imported from folder, everyday a new CSV file is coming into my folder. I have problems if a new column is added in my source CSV, especially when it is not located at the end of the table.

I used to deal with that by just making sure my CSV structure would not change, but I'm starting this project again and want to anticipate this to happen.

 

I tried changing the number of columns in my sample file in advanced editor, but I guess the problem is that my Sample File is one without that additional column, so it just adds an empty column at the end. Maybe I should change the path of my Sample File, to pick the file with the highest number of columns? I don't know how to do that, and I'm thinking that then the problem would be on the other files, that would be then missing columns.

 

Any ideas?

 

Thanks a lot in advance

 

Thomas

Need Help

3 Replies

  • HotChilli's avatar
    HotChilli
    Icon for Community Champion rankCommunity Champion
    2 years ago

    There is a parameter in Csv.Document which is for the number of columns (this is probably the Source step in Power Query). You can remove this parameter.

    --

    You will need to think about other transforms that may break if you do this. If the rest of the transforms are simple or involve Unpivotting to cope with multiple, similar columns, it should be ok.

    • ThomasRichard21's avatar
      ThomasRichard21
      Frequent Visitor
      2 years ago

      Hi HotChilli thanks for the fast answer.

      How exactly can I remove this parameter? In my query editor, on advanced editor? I can't find Csv.document