Forum Discussion
Merge tables using wildcards
I have a small fact table of about 10,000 records. Based on 5 columns in each row, I need to add 3 extra columns from a small dimension table of around 70 records.
The problem is that the rules in the dimension table include wildcards. The Reference column in the dim table uses prefixes to match. So Fact References of 20123 and 20999 need to match the Dim Reference of 20*. Other columns need a wildcard to match any value, so Fact assignment of A, B, or Z would match a Dim Assignment of *.
In SQL, I can do a join with wildcards:
WHERE fact.company LIKE dim.company + '%'
AND fact.reference LIKE dim.reference+'%'
AND fact.assignment LIKE dim.assignment+'%'
AND fact.plant LIKE dim.plant+'%'
AND fact.customer LIKE dim.customer+'%'
I haven't figured out any way to do this in Power Query. Chris Webb has a Power Query LIKE function, but it looks like it would only work to filter a single column. I need to join on 5 columns. I also found a post that matched on multiple columns using Table.MatchesAnyRows, but it simply indicated where there was a match without returning any columns.
Here is sample data:
FACT
| Company | Assignment | Reference | Plant | Customer |
| A | null | 20123 | J | 5 |
| A | null | 20999 | J | 5 |
| A | Z | M123 | J | null |
| B | CMIX | CMIX | G | null |
| Z | null | 99999 | J | 5 |
DIM
| Company | Reference | Assignment | Plant | Customer | Result1 | Result2 | Result3 |
| A | 20* | null | J | 5 | G | P | G |
| A | 21* | G | J | null | G | G | P |
| A | C M* | * | J | null | Z | P | O |
| A | M* | * | J | null | Z | O | P |
| B | 10* | null | D | 8 | E | L | I |
| * | ADJ* | * | * | * | null | null | null |
| * | CMI* | * | * | * | null | null | null |
The goal is to match records here to the fact table and add the 3 result columns. If there is no match, I need to identify those rows as grief, either in the output or in a separate query. It would be acceptable to return nulls if no match.
The * character in the Dim table indicates match of 0 or more characters (needs to match null).
All values in the reference column are prefixes with the wildcard at the end.
The * character could be changed to % or anything else as needed or removed.
Desired output
| Company | Assignment | Reference | Plant | Customer | Result1 | Result2 | Result3 |
| A | null | 20123 | J | 5 | G | P | G |
| A | null | 20999 | J | 5 | G | P | G |
| A | Z | M123 | J | null | Z | O | P |
| B | CMIX | CMIX | G | null | null | null | null |
| Z | null | 99999 | J | 5 | grief | grief | grief |
I have a sample file I could attach if I can figure out how!
Thanks for any help!
let Fact = Excel.CurrentWorkbook(){[Name="Fact"]}[Content], Dim = Excel.CurrentWorkbook(){[Name="Dim"]}[Content], fx=(t1,t2)=> let a=Text.From(t2), b=Text.From(t1), c=if Text.End(a,1)="*" then Text.StartsWith(b??"",Text.Start(a,Text.Length(a)-1)) else a=b in if a="*" then true else c, Custom1 = Table.AddColumn( Fact, "n", each let col=Table.ColumnNames(Fact), b=Table.SelectRows( Dim, (x)=>List.AllTrue(List.Transform(col,(y)=>fx(Record.Field(_,y),Record.Field(x,y)))) ) in if b{0}?=null then {null,null,null} else Record.ToList(Record.SelectFields(b{0},{"Result1","Result2","Result3"})) ), Custom2 = Table.SplitColumn(Custom1,"n",each _,{"Result1","Result2","Result3"}) in Custom2
1 Reply
let Fact = Excel.CurrentWorkbook(){[Name="Fact"]}[Content], Dim = Excel.CurrentWorkbook(){[Name="Dim"]}[Content], fx=(t1,t2)=> let a=Text.From(t2), b=Text.From(t1), c=if Text.End(a,1)="*" then Text.StartsWith(b??"",Text.Start(a,Text.Length(a)-1)) else a=b in if a="*" then true else c, Custom1 = Table.AddColumn( Fact, "n", each let col=Table.ColumnNames(Fact), b=Table.SelectRows( Dim, (x)=>List.AllTrue(List.Transform(col,(y)=>fx(Record.Field(_,y),Record.Field(x,y)))) ) in if b{0}?=null then {null,null,null} else Record.ToList(Record.SelectFields(b{0},{"Result1","Result2","Result3"})) ), Custom2 = Table.SplitColumn(Custom1,"n",each _,{"Result1","Result2","Result3"}) in Custom2