Forum Discussion

justlogmein's avatar
justlogmein
Helper III
4 years ago
Solved

How to combine csv files with inconsistent columns from within a folder?

I want to use the From Folder feature to combine csv files that have different some different columns. The columns are not just different names, but also contain different data.   If I manually add...
  • KT_Bsmart2gethe's avatar
    4 years ago

    Hi justlogmein ,

     

    I have downloaded your files and create the code below. you can copy and paste into your dashboard file sample data query in advance editor.

     

    See below modified code (Import from Folder):

     

    let

     

    //You can replace the file with your drill down path to the blue text below
    Source = Folder.Files("C:\Users\cktan\Documents\Solutions\justlogmein\Sample Data"),

     

    // Add new column to get the files and promote headers before expand the columns
    GetTables = Table.AddColumn(Source, "GetTbl", each Table.PromoteHeaders(Csv.Document([Content],

    [Delimiter=",", Columns=5, Encoding=1252, QuoteStyle=QuoteStyle.None]))),

     

    //Below code is to get the column names from each csv file and combine them as a list
    GetColumnNames = Table.AddColumn(GetTables, "GetColName", each Table.ColumnNames([GetTbl])),
    ColNameList = Table.Distinct(Table.ExpandListColumn(Table.SelectColumns(GetColumnNames,{"GetColName"}), "GetColName"))[GetColName],

     

    //Keep the name and added column before we expand
    Filtered_Columns = Table.SelectColumns(GetTables,{"Name", "GetTbl"}),

     

    //the above column name list is use at below code in blue text to allow dynamic expand
    #"Expanded GetTbl" = Table.ExpandTableColumn(Filtered_Columns, "GetTbl", ColNameList)


    in
    #"Expanded GetTbl"

     

    I hope this helps