Forum Discussion
power query - loading files from different file paths
- 1 year ago
thanks again PwerQueryKees.
i also noticed that the table names are auto-generated for different years when i created them individually separately for each year. each file only has 1 table.
i'll try your solution as soon as i get on my computer later & revert as necessary.
krgds, -nik
- 1 year ago
Hi hood2media ,
Thank you for reaching out to Microsoft fabric community.
Try these steps
Create ParametersNavigate to Home > Manage Parameters > New Parameter and set up the following:
YearFrom – type: Number (e.g., 2016)
YearTo – type: Number (e.g., 2018)
Station – type: Text (e.g., JFK)
You can later link these to slicers or keep the default values.
Use this M codelet
// PARAMETERS
StartYear = YearFrom,
EndYear = YearTo,
Station = Station,
BasePath = "C:\Users\nik\H2M\H2M - us.data\data\",
// Create list of years (2016, 2017, 2018)
YearList = List.Numbers(StartYear, EndYear - StartYear + 1),
// Build folder paths
FolderPaths = List.Transform(YearList, each BasePath & Text.From(_) & "\" & Station),
// Function to get .xlsx files from a folder
GetFilesFromFolder = (folderPath as text) =>
let
Files = Folder.Files(folderPath),
ExcelFiles = Table.SelectRows(Files, each Text.EndsWith([Extension], ".xlsx"))
in
ExcelFiles,
// Loop through folders and collect Excel file metadata
AllFilesTables = List.Transform(FolderPaths, each try GetFilesFromFolder(_) otherwise null),
NonNullTables = List.RemoveNulls(AllFilesTables),
AllFiles = Table.Combine(NonNullTables),
// Load actual Excel content from each file
AddContent = Table.AddColumn(AllFiles, "Data", each Excel.Workbook(File.Contents([Folder Path] & [Name]), true)),
// Expand the first table in the Excel file
Expanded = Table.ExpandTableColumn(AddContent, "Data", {"Data"}, {"Data"}),
// Combine all data into one table
FinalCombined = Table.Combine(Expanded[Data])
in
FinalCombinedApply and Load
Select Close & Apply (top left corner) to load the query to the Power BI Data Model.
Create Table Visual
In the Fields pane, expand CombinedData and drag in the following fields:
FlightID
Departure
Arrival
Status
This will show the combined flight data based on your selected parameters.
Please find the below attached .pbix file for your reference.
Regards,
Sreeteja
hood2media Try using
let
// Define parameters
YearFrom = 2016,
YearTo = 2018,
Station = "JFK",
// Generate list of years
Years = List.Numbers(YearFrom, YearTo - YearFrom + 1),
// Generate list of file paths
FilePaths = List.Transform(Years, each "C:\Users\nik\H2M\H2M - us.data\data\" & Text.From(_) & "\" & Station),
// Function to get files from a folder
GetFilesFromFolder = (folderPath as text) =>
let
Source = Folder.Files(folderPath),
FilteredFiles = Table.SelectRows(Source, each Text.EndsWith([Name], ".xlsx"))
in
FilteredFiles,
// Get all files from the generated file paths
AllFiles = List.Combine(List.Transform(FilePaths, each GetFilesFromFolder(_))),
// Load and combine the Excel files
CombineExcelFiles = Table.Combine(
List.Transform(AllFiles[Content], each Excel.Workbook(_, null, true)[Data])
)
in
CombineExcelFiles
hi bhanu_gautam,
thanks for responding.
it seems that all is fine till the function (defined as 'FilePaths') to generate list of file paths .
the process stops at the function (defined as 'GetFilesFromFolder') to get files from the folder(s). it invokes a request to enter parameter for folderPath which i believe should come from the previous function called 'FilePaths'.
how do i proceed?
kind rgds, -nik