Forum Discussion

mmpbi's avatar
mmpbi
Regular Visitor
5 years ago

Duplicate column in the rowset error when appending files

I wrote the folowing power-query script. It calls fnGetContents, which is a custom func that removes the first 8 lines of every file (since they contain all sorts of crap I don't need). Then it expan...
Need Help
Anonymous's avatar
Anonymous
Not applicable
5 years ago

Hi mmpbi 

 

In you custom function, the custom column has all tables, right? Have you checked all headers in each table? Besides, in the list, try Trim and/or Text.Lower (Upper/Proper) before List.Distinct

  • mmpbi's avatar
    mmpbi
    Regular Visitor
    5 years ago

    Thanks for your reply! The point is that the headers in most tables are the same, but there are some tables that are gonna have different headers. I want the script to append to the existing whenever a column was already found in one of the previous tables, or create a new column when it didn't yet.

    • Anonymous's avatar
      Anonymous
      Not applicable
      5 years ago

      Hi mmpbi 

       

      I understand your code, I have done similar one to load multiple files with different headers, but have not seen this error...can you try this, if you don't like Lowercase, Proper or Upper:

       

       

      (filepath)=>
let
    Source = Csv.Document(File.Contents((filepath)),[Delimiter=";",  Encoding=65001, QuoteStyle=QuoteStyle.None]),
   #"Removed Top Rows" = Table.Skip(Source ,8),
    #"Promoted Headers" = Table.PromoteHeaders(#"Removed Top Rows", [PromoteAllScalars=true]),
   custom = Table.TransformColumnNames( Table.TransformColumnNames( #"Promoted Headers", Text.Trim), Text.Lower)
in
    Custom