Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
IAM
Helper III
Helper III

SQL connection -> different Query stages.. Why???

Hello,

 

I am working on a report, made by someone I cannot reach anymore. I would like to know the logic behind this setup.. 

 

There are (very large) datatables connected with SQL.

In Power Query he has divided different queries into folders/sections:

 

IAM_0-1657693698610.png

In each 'stage' are some basic steps, like changing a column name and type and merging with lookup tables.

 

Could it be that he needed to make some adjustments first on two tables and then merge them to the other?

If so, is this the best approach? 

 

It takes about 20 to 30 minutes to refresh the dataset. Does it take this long because of this setup?

 

I hope someone can enlighten me 🙂

IAM_1-1657694318193.png

 

 

1 ACCEPTED SOLUTION
indkitty
Helper II
Helper II

Folder or Data source should not be affecting. The SQL data may be cleansed or Raw data, at times I have used both cleansed and raw data. 

I have data refresh running more than 30 minutes, that would be during initial stage. 

Check if the merge table is at the last process, if they are then the new table is dependent on other tables data refresh. You could also check if the data has to be refreshed for the entire tables or update /append only the new data into the table. This would use incremental refresh which is much faster. 

Guysinacube has tutorials for Query folding, incremental refresh, reduction of data by using parameters. 

 

View solution in original post

2 REPLIES 2
indkitty
Helper II
Helper II

Folder or Data source should not be affecting. The SQL data may be cleansed or Raw data, at times I have used both cleansed and raw data. 

I have data refresh running more than 30 minutes, that would be during initial stage. 

Check if the merge table is at the last process, if they are then the new table is dependent on other tables data refresh. You could also check if the data has to be refreshed for the entire tables or update /append only the new data into the table. This would use incremental refresh which is much faster. 

Guysinacube has tutorials for Query folding, incremental refresh, reduction of data by using parameters. 

 

Vijay_A_Verma
Super User
Super User

The folder like organization doesn't impact queries. It's a way to organize it visually and will not have any impact on loading time. 

Why this is taking half an hour - You will need to analyze the relevent queries and their inter-relationships (how these are placed on left side need to discarded from your consideration)

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.

Power BI Header Change

Power BI Header Update

Read about the exciting updates for the Power BI forum.

Top Solution Authors
Top Kudoed Authors