Forum Discussion

Jqarroyo's avatar
Jqarroyo
Frequent Visitor
7 years ago

Fuzzy Matching - Scores & Size limits

Hello,


I'm testing the fuzzy matching merge option and got a couple of queries:

 

- Is there any way of getting the scores on the output returned? The Fuzzy lookup addon (Excel) had this possibility which I think is very convenient in case you want to review the accuracy of the results over the similarity threshold.

- Have you tried this tool on big data sets? I'm quite impressed on how the algorithm is working with a couple of small examples I've tested but seems is taking ages to run for a bigger connection (1M rows approx.) Any parameter I can change to make it work faster?

 

The logic behind this tool is something that could have a very positive impact on one of the projects I'm working so any experience, feedback on it would be more than welcome!

 

Many thanks, 

 

Jq

Need Help
Tips and Tricks

2 Replies

    • Jqarroyo's avatar
      Jqarroyo
      Frequent Visitor
      6 years ago

       

      Hi v-eachen-msft 


      Thanks for your response.

      I've got the August release of Power Bi Desktop so I believe it should contain the most recent version of Fuzzy Matching.


      My connections/tables are quite clean as I covered anything re filters/trim/cleaning etc in a different model so still don't get my head around the slow performance...is it 1M-2M rows too much to run this tool? I’m running a I7 with 16Gb RAM so it’s not a very slow machine.

      The excel add-on (Fuzzy Lookup) is slow and obviously has the limitations of the size...but I would expect "Fuzzy Matching" via Power BI to perform faster and better in larger data sets..

      I'm going to have a look to that post in more detail. Very interesting!

      Thanks again for your help.

       

      Jq