Forum Discussion
Get all combined subDirectories with Folder.Contents() from multiple servers (mainDirectories),
No, you need to bring out the big guns. AD extracts, Purview, Scanner API. Low code/no code tools tend to be overwhelmed when confronted with massive amounts of data.
possible value relevance: Most companies are SMEs (future value/cashflow). Most SMEs are not fully digitalized.
Many SMEs are MS-clients, still running file-servers, many run RDBs, fewer RDFs or kafka, i.e. data is stored in files.
The wish is to have a stackable select-ETL (until full digitalization, if at all) to select/access file-data via PBI (PQ/PP).
AIM: Enable any user to browse & combi-select multiple subdirectories to query file-content with separate ETLs.
Regards
You may want to continue exploring other tools. While Power Query is very powerful and flexible it is not a "proper" ETL tool. Its sole aim is to prepare data so that it can be used by Vertipaq. There was never a design goal to load anything elsewhere.
Thank you for the VertiPaq hint, indeed helpful/new.
You may want to continue exploring other tools?
It means power platform apps/page/automate/va, eventhub/dataverse...?
Coming py/R/SQL/git, MS strongly adviced no/low-Code power platform.No, you need to bring out the big guns. AD extracts, Purview, Scanner API. Low code/no code tools tend to be overwhelmed when confronted with massive amounts of data.
Note on the concrete task to be accomplished that appears to be a standard BI use case
Task: Load data from distributed XLS-files (of same kind) from changing (renamed/moved) folders
via PBI (PQ) into a flattened MS-Cube to consolidate file-data for comparative pivot-analysis.Task: Load data from distributed XLS-files (of same kind) from changing (renamed/moved) folders via PBI (PQ) into a flattened MS-Cube to consolidate file-data for comparative pivot-analysis.This is from when? 2013 ?