Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
IAM
Helper III
Helper III

SQL connection -> different Query stages.. Why???

Hello,

 

I am working on a report, made by someone I cannot reach anymore. I would like to know the logic behind this setup.. 

 

There are (very large) datatables connected with SQL.

In Power Query he has divided different queries into folders/sections:

 

IAM_0-1657693698610.png

In each 'stage' are some basic steps, like changing a column name and type and merging with lookup tables.

 

Could it be that he needed to make some adjustments first on two tables and then merge them to the other?

If so, is this the best approach? 

 

It takes about 20 to 30 minutes to refresh the dataset. Does it take this long because of this setup?

 

I hope someone can enlighten me 🙂

IAM_1-1657694318193.png

 

 

1 ACCEPTED SOLUTION
indkitty
Helper II
Helper II

Folder or Data source should not be affecting. The SQL data may be cleansed or Raw data, at times I have used both cleansed and raw data. 

I have data refresh running more than 30 minutes, that would be during initial stage. 

Check if the merge table is at the last process, if they are then the new table is dependent on other tables data refresh. You could also check if the data has to be refreshed for the entire tables or update /append only the new data into the table. This would use incremental refresh which is much faster. 

Guysinacube has tutorials for Query folding, incremental refresh, reduction of data by using parameters. 

 

View solution in original post

2 REPLIES 2
indkitty
Helper II
Helper II

Folder or Data source should not be affecting. The SQL data may be cleansed or Raw data, at times I have used both cleansed and raw data. 

I have data refresh running more than 30 minutes, that would be during initial stage. 

Check if the merge table is at the last process, if they are then the new table is dependent on other tables data refresh. You could also check if the data has to be refreshed for the entire tables or update /append only the new data into the table. This would use incremental refresh which is much faster. 

Guysinacube has tutorials for Query folding, incremental refresh, reduction of data by using parameters. 

 

Vijay_A_Verma
Super User
Super User

The folder like organization doesn't impact queries. It's a way to organize it visually and will not have any impact on loading time. 

Why this is taking half an hour - You will need to analyze the relevent queries and their inter-relationships (how these are placed on left side need to discarded from your consideration)

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayPowerBICarousel

Fabric Monthly Update - May 2024

Check out the May 2024 Fabric update to learn about new features.

LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.

Top Solution Authors
Top Kudoed Authors