Forum Discussion
Merging with one exact match column and one fuzzy join column
- 4 years ago
Hi ianbruckner ,
I'm afraid, but the merge/join-condition can only be set on the join level and not on column-level.For inner joins, you can simply do them one after another, but for outer joins you have to expand the matched columns and then apply some logic afterwards (taking only those rows, where both expansions returned values).
I know this is old, but I came across this while searching for a solution that would work for a similar situation, but in my case I needed to limit the fuzzy match results to just the top result. I was inspired by LaasyaS's suggestion to group by first. However, I couldn't perform the fuzzy match first as I needed a single fuzzy match result from among the exact match options. Thus, here is the final solution I found works very cleanly:
1) First, group your starting table (whichever one will include ALL records generally, but this will be the table that is not restricted to the n number of fuzzy match results) by all the join columns for EXACT matching. For the aggregation, select "All Rows" (I generally name this column "Data" and will going forward). This will leave you with all exact join columns and a column composed of sub-tables.
2) Left-join onto the other table using exact matching. This will give you a 2nd column of tables, each a subtable of all the exact-match options for your joining.
3) Finally, add a custom additional column and for the formula, perform the fuzzy join with the matches limit and everything. You can ignore the exact-match columns if you want, since those will be redundant, but this will perform a fuzzy join for each subset of the data partitioned by your exact match columns.
4) You can delete all the columns for the source tables and then expand your newest column to get all the data joined as desired. Here's an example of that with 2 tables called "Employees" and "Positions" where several companies' position definitions are fuzzy matched to job titles for employees missing their position code, but only using their own company's options.
Employees (ID | Name | Title | Company ID | Position ID)
Positions (ID | Position Desc | Company ID)
let
Source = Table.SelectRows(Employees, each [Position ID] = null),
#"Grouped Rows" = Table.Group(Source, {"Company ID"}, {{"Data", each _, type table [ID=nullable text, Name=nullable text, Title=nullable text, Company ID=nullable text, Position ID=nullable text]}}),
#"Merged Queries" = Table.NestedJoin(#"Grouped Rows", {"Company ID"}, Positions, {"Company ID"}, "Positions", JoinKind.LeftOuter),
#"Added Custom" = Table.AddColumn(#"Merged Queries", "Company Positions", each Table.FuzzyNestedJoin([Data], {"Title"}, [Positions], {"Position Desc"}, "matches", JoinKind.LeftOuter, [SimilarityThreshold=0.8, IgnoreCase=true, IgnoreSpace=true, NumberOfMatches=1]), type table),
#"Removed Other Columns" = Table.SelectColumns(#"Added Custom",{"Company Positions"}),
#"Expanded Company Positions" = Table.ExpandTableColumn(#"Removed Other Columns", "Company Positions", {"ID", "Name", "Title", "Company ID", "matches"}, {"ID", "Name", "Title", "Company ID", "matches"}),
#"Expanded matches" = Table.ExpandTableColumn(#"Expanded Company Positions", "matches", {"ID", "Position Desc", "Company ID"}, {"matches.ID", "matches.Position ID", "matches.Company ID"})
in
#"Expanded Matches"
Hopefully that is helpful to someone in the future!