Forum Discussion
Getting data from CSV files that have slightly different columns each week
- 6 years ago
Hi Anonymous
Try this. It's basically loading the files from the folder, transposing the tables, eliminating duplicates and transposing again
let Source = Folder.Files(FOLDER_WHERE_YOUR_DATA_IS), #"Added Custom" = Table.AddColumn(Source, "Custom", each Csv.Document([Content])), #"Added Custom1" = Table.AddColumn(#"Added Custom", "Custom.1", each Table.Transpose([Custom])), #"Removed Other Columns" = Table.SelectColumns(#"Added Custom1",{"Custom.1"}), #"Expanded Custom.1" = Table.ExpandTableColumn(#"Removed Other Columns", "Custom.1", {"Column1", "Column2", "Column3", "Column4"}, {"Column1", "Column2", "Column3", "Column4"}), #"Removed Duplicates" = Table.Distinct(#"Expanded Custom.1", {"Column1"}), #"Transposed Table" = Table.Transpose(#"Removed Duplicates"), #"Promoted Headers" = Table.PromoteHeaders(#"Transposed Table", [PromoteAllScalars=true]), #"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"Job", type text}, {"WW31", Int64.Type}, {"WW32", Int64.Type}, {"WW33", Int64.Type}, {"WW34", Int64.Type}, {"WW35", Int64.Type}}) in #"Changed Type"Please mark the question solved when done and consider giving kudos if posts are helpful.
Contact me privately for support with any larger-scale BI needs, tutoring, etc.
Cheers
I would create a function (or modify the Transform Example File query) to unpivot all the Week columns, append the files together, and then remove rows with duplicate Week and Job. You could then pivot it back out, but I recommend you keep it unpivoted.
If this works for you, please mark it as the solution. Kudos are appreciated too. Please let me know if not.
Regards,
Pat
- Anonymous6 years agoNot applicable
Can you perhaps show me with an example? I'm pretty new to this and not quite sure what you mean.