Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

The ultimate Microsoft Fabric, Power BI, Azure AI & SQL learning event! Join us in Las Vegas from March 26-28, 2024. Use code MSCUST for a $100 discount. Register Now

Reply
campelliann
Post Patron
Post Patron

List.PositionOf > performance than Merge Queries?

Hi there, 

As Merges between Tables in Power Query are considerer "bad" for performance, I have been trying to use List.PositionOf to get the index of a key on another table (a little bit like index match on excel). Is this better than merges to do a "Vlookup" on another table?

 

The logic is something like this you add a column and then OutsideTableColumn{List.PositionOf(IDsOutsideTable,IDsInsideTable})

This video explains it

 https://www.youtube.com/watch?v=U97r2_O8IyI

Thanks


1 ACCEPTED SOLUTION
ImkeF
Super User
Super User

Hi @campelliann ,
you're probably asking that question here because you didn't experience the performance advantage of the solution in that video?

I know that Power Query can be a bit funny sometimes when it comes to performance, but I would really like to see the data where the method presented in the video is actually faster than a merge. Reason for that is that the lookup- table will be referenced once per each row in the (fact) table while with a merge, it would only be referenced once. To improve performance in that case, one could try using a Table.Buffer on the lookup-table to prohibit multiple referencing. But still then I would be very surprised if the index is actually faster. Because again, with that method you have to reference the lookup table at least twice: Once to retrieve the index and second to retrieve the actual lookup value.
To my experience, a merge will likely be the fastest operation.
With exact matches on columns that have unique values in the lookup-table you can speed up performance immensly if you transform that merging column to a key-column like described here: Chris Webb's BI Blog: Improving The Performance Of Aggregation After A Merge In Power BI And Excel P...

 

I would like to see other sources that made you believe that merges slow down performance (in comparison to other methods where you have to lookup data from other tables).

With regard to aproximate matches, I recommend this method: (3) VLOOKUP-True Equivalent in Power Query with Dynamic Tiers - YouTube


Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

View solution in original post

1 REPLY 1
ImkeF
Super User
Super User

Hi @campelliann ,
you're probably asking that question here because you didn't experience the performance advantage of the solution in that video?

I know that Power Query can be a bit funny sometimes when it comes to performance, but I would really like to see the data where the method presented in the video is actually faster than a merge. Reason for that is that the lookup- table will be referenced once per each row in the (fact) table while with a merge, it would only be referenced once. To improve performance in that case, one could try using a Table.Buffer on the lookup-table to prohibit multiple referencing. But still then I would be very surprised if the index is actually faster. Because again, with that method you have to reference the lookup table at least twice: Once to retrieve the index and second to retrieve the actual lookup value.
To my experience, a merge will likely be the fastest operation.
With exact matches on columns that have unique values in the lookup-table you can speed up performance immensly if you transform that merging column to a key-column like described here: Chris Webb's BI Blog: Improving The Performance Of Aggregation After A Merge In Power BI And Excel P...

 

I would like to see other sources that made you believe that merges slow down performance (in comparison to other methods where you have to lookup data from other tables).

With regard to aproximate matches, I recommend this method: (3) VLOOKUP-True Equivalent in Power Query with Dynamic Tiers - YouTube


Imke Feldmann (The BIccountant)

If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!

How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries

Helpful resources

Announcements
Fabric Community Conference

Microsoft Fabric Community Conference

Join us at our first-ever Microsoft Fabric Community Conference, March 26-28, 2024 in Las Vegas with 100+ sessions by community experts and Microsoft engineering.

February 2024 Update Carousel

Power BI Monthly Update - February 2024

Check out the February 2024 Power BI update to learn about new features.

Fabric Career Hub

Microsoft Fabric Career Hub

Explore career paths and learn resources in Fabric.

Fabric Partner Community

Microsoft Fabric Partner Community

Engage with the Fabric engineering team, hear of product updates, business opportunities, and resources in the Fabric Partner Community.