Forum Discussion

CAMA_UK_18's avatar
CAMA_UK_18
Frequent Visitor
4 years ago
Solved

Small datasets, many queries/steps or large data sets, fewer queries/steps?

Hi What would you say is the optimal set-up for the scenario below?   I have 10 customers (say). For each of those customers, there are three tables (all exactly the same structures) on separate t...
  • MarkLaf's avatar
    4 years ago

    In my experience working with one Excel file with multiple tables of same structure, I generally get best performance by immediately expanding (and automatically combining) tables off the initial Excel.Workbook access function. Pre-reqs/steps:

    1. Make sure inside your Excel workbook that your data tables are formatted as tables. This let's you skip any need to do any transforms before combining tables (namely, Table.PromoteHeaders). They show up as Kind = Table in the access output
    2. From the initial access step (e.g. Excel.Workbook( Web.Contents( [workbook url] ) ) ), filter the table of content to just your tables (not sheets) of same-structure data.
    3. Now, you can simply expand the Data column to get your combined table. You'll have the added benefit of keeping a "Source Table" column with the Name or Item column (others can be removed before expanding)

    I've found that this minimizes repeated queries to the data source, and makes maintenance easier (e.g. schema drift like when column name changes; can just fix in the one expand step rather than multiple places)