SQL connection -> different Query stages.. Why???

[IAM]

Hello,

I am working on a report, made by someone I cannot reach anymore. I would like to know the logic behind this setup.. 

There are (very large) datatables connected with SQL.

In Power Query he has divided different queries into folders/sections:

In each 'stage' are some basic steps, like changing a column name and type and merging with lookup tables.

Could it be that he needed to make some adjustments first on two tables and then merge them to the other?

If so, is this the best approach? 

It takes about 20 to 30 minutes to refresh the dataset. Does it take this long because of this setup?

I hope someone can enlighten me 🙂

[indkitty · Accepted solution]

Folder or Data source should not be affecting. The SQL data may be cleansed or Raw data, at times I have used both cleansed and raw data. 

I have data refresh running more than 30 minutes, that would be during initial stage. 

Check if the merge table is at the last process, if they are then the new table is dependent on other tables data refresh. You could also check if the data has to be refreshed for the entire tables or update /append only the new data into the table. This would use incremental refresh which is much faster. 

Guysinacube has tutorials for Query folding, incremental refresh, reduction of data by using parameters. 

[Vijay_A_Verma]

The folder like organization doesn't impact queries. It's a way to organize it visually and will not have any impact on loading time. 

Why this is taking half an hour - You will need to analyze the relevent queries and their inter-relationships (how these are placed on left side need to discarded from your consideration)