Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
4 years ago

Tables taking too long to merge in Power Query

Hi,   I have 2 tables of approximately 1.5L records from MySQL. I need to merge the tables based on a key(Text datatype) which is unique. The merging takes very long time to load and never comple...
Need Help
BA_Pete's avatar
BA_Pete
Icon for Super User rankSuper User
4 years ago

Hi Anonymous ,

 

Vijay has given you a good steer on how to speed up your merges, but I'd recommend not merging at all, if possible.

Merges are a 'whole table' operation in Power Query i.e. they require an entire table to be loaded into memory (or at least every row to be scanned at least once), so can hit memory limits for very large tables.

 

I would recommend just sending both tables to the data model and relating them on your unique key field. You may see an increase in your PBIX file size (as relationships have a 'size' in PBI), but the VertiPaq engine/DAX should be orders of magnitude faster than a Power Query merge.

 

Pete