Forum Discussion
Sharepoint Data Import - Multiple File Processing
Hi Las ,
It depends on whether all the files that you are getting from the different folders are the same structure and data type as one another.
If you're keeping different types of files in different folders, then you will need a set of helper queries for each different folder, as the transformations that will need to be performed on the files in each folder will be different.
If the files in all folders are all the same structure/data type, then I would recommend throwing them all into one folder so you can manage the whole lot with a single set of helper queries. If you want to keep the folder structure locally for eas of reference, you may be able to use Power Automate to automatically copy new entries into a combined folder for you.
If you try to put files of different structure and data type into one folder and run them all through the Combine & Transform process, you will get errors or, at the very least, unexpected and/or unreliable output.
Pete
Hi BA_Pete ,
Thank you for your detailed response and really appreciate that.
When creating a connection to Sharepoint Folder to extract a file / folder it always create a helper query for each query. How should i better manage these queries? Is there a way to use one helper query for all the files? So that its neat and clean in the power query pane.
Thank you.
Las
Super UserHi Las ,
It depends on whether all the files that you are getting from the different folders are the same structure and data type as one another.
If you're keeping different types of files in different folders, then you will need a set of helper queries for each different folder, as the transformations that will need to be performed on the files in each folder will be different.
If the files in all folders are all the same structure/data type, then I would recommend throwing them all into one folder so you can manage the whole lot with a single set of helper queries. If you want to keep the folder structure locally for eas of reference, you may be able to use Power Automate to automatically copy new entries into a combined folder for you.
If you try to put files of different structure and data type into one folder and run them all through the Combine & Transform process, you will get errors or, at the very least, unexpected and/or unreliable output.
Pete