Forum Discussion
How to combine csv files with inconsistent columns from within a folder?
- 4 years ago
Hi justlogmein ,
I have downloaded your files and create the code below. you can copy and paste into your dashboard file sample data query in advance editor.
See below modified code (Import from Folder):
let
//You can replace the file with your drill down path to the blue text below
Source = Folder.Files("C:\Users\cktan\Documents\Solutions\justlogmein\Sample Data"),// Add new column to get the files and promote headers before expand the columns
GetTables = Table.AddColumn(Source, "GetTbl", each Table.PromoteHeaders(Csv.Document([Content],[Delimiter=",", Columns=5, Encoding=1252, QuoteStyle=QuoteStyle.None]))),
//Below code is to get the column names from each csv file and combine them as a list
GetColumnNames = Table.AddColumn(GetTables, "GetColName", each Table.ColumnNames([GetTbl])),
ColNameList = Table.Distinct(Table.ExpandListColumn(Table.SelectColumns(GetColumnNames,{"GetColName"}), "GetColName"))[GetColName],//Keep the name and added column before we expand
Filtered_Columns = Table.SelectColumns(GetTables,{"Name", "GetTbl"}),//the above column name list is use at below code in blue text to allow dynamic expand
#"Expanded GetTbl" = Table.ExpandTableColumn(Filtered_Columns, "GetTbl", ColNameList)
in
#"Expanded GetTbl"I hope this helps
Hi justlogmein ,
I have downloaded your files and create the code below. you can copy and paste into your dashboard file sample data query in advance editor.
See below modified code (Import from Folder):
let
//You can replace the file with your drill down path to the blue text below
Source = Folder.Files("C:\Users\cktan\Documents\Solutions\justlogmein\Sample Data"),
// Add new column to get the files and promote headers before expand the columns
GetTables = Table.AddColumn(Source, "GetTbl", each Table.PromoteHeaders(Csv.Document([Content],
[Delimiter=",", Columns=5, Encoding=1252, QuoteStyle=QuoteStyle.None]))),
//Below code is to get the column names from each csv file and combine them as a list
GetColumnNames = Table.AddColumn(GetTables, "GetColName", each Table.ColumnNames([GetTbl])),
ColNameList = Table.Distinct(Table.ExpandListColumn(Table.SelectColumns(GetColumnNames,{"GetColName"}), "GetColName"))[GetColName],
//Keep the name and added column before we expand
Filtered_Columns = Table.SelectColumns(GetTables,{"Name", "GetTbl"}),
//the above column name list is use at below code in blue text to allow dynamic expand
#"Expanded GetTbl" = Table.ExpandTableColumn(Filtered_Columns, "GetTbl", ColNameList)
in
#"Expanded GetTbl"
I hope this helps
- justlogmein4 years agoHelper III
Thank you, this solution worked and the comments helped a lot.
- karimm4 years agoHelper III
Thank you so much for this.
I managed to combine several Excel files with different structures using your approach (with some tweaks).
Your solution on csv encouraged me to push the limits and try to resolve it!