Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
Anonymous
Not applicable

Tables taking too long to merge in Power Query

Hi,

 

I have 2 tables of approximately 1.5L records from MySQL.

I need to merge the tables based on a key(Text datatype) which is unique.

The merging takes very long time to load and never completes it. (Refer below image).

Prajna_1-1651639036782.png

 

Any solution to the above problem will be really helpful.

 

Thanks,

Prajna

 

2 REPLIES 2
BA_Pete
Super User
Super User

Hi @Anonymous ,

 

Vijay has given you a good steer on how to speed up your merges, but I'd recommend not merging at all, if possible.

Merges are a 'whole table' operation in Power Query i.e. they require an entire table to be loaded into memory (or at least every row to be scanned at least once), so can hit memory limits for very large tables.

 

I would recommend just sending both tables to the data model and relating them on your unique key field. You may see an increase in your PBIX file size (as relationships have a 'size' in PBI), but the VertiPaq engine/DAX should be orders of magnitude faster than a Power Query merge.

 

Pete



Now accepting Kudos! If my post helped you, why not give it a thumbs-up?

Proud to be a Datanaut!




Vijay_A_Verma
Super User
Super User

As first level of performance improvement, do following -

1. Remove all columns which are not required for your analysis / decision and not required for merging in botht the tables.

2. Also if you can apply filter to minimize number of rows required to do merging.

3. File - Options & settings - Options - Current File - Data load -Uncheck "Allow data previews to download in the background"

4. You can also explore possibility of using Table.Join as given here - https://blog.crossjoin.co.uk/2020/06/07/optimising-the-performance-of-power-query-merges-in-power-bi... 

Helpful resources

Announcements
July 2024 Power BI Update

Power BI Monthly Update - July 2024

Check out the July 2024 Power BI update to learn about new features.

July Newsletter

Fabric Community Update - July 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors