Forum Discussion
How to extract continuous string set from a column with multiple criteria
Anonymous - OK, I updated this with a Client Name output version and Project output version in 2 additional columns. I checked the matches, don't see anything obviously amiss. Solved?
PBIX is attached.
Anonymous Except it doesn't work that way. To implement the original algorithm in DAX I had to do this:
Let's take just the first 2 client names
| 1 | Aaron's |
| 2 | ABB (CA VDA) |
In order to emulate looping, which is impossible in DAX, this becomes:
| 1 | 1 | A |
| 1 | 2 | Aa |
| 1 | 3 | Aar |
| 1 | 4 | Aaro |
| 1 | 5 | Aaron |
| 1 | 6 | Aaron' |
| 1 | 7 | Aaron's |
| 2 | 1 | A |
| 2 | 2 | AB |
| 2 | 3 | ABB |
and so on, the entire list of projects gets blown out into a big table like this. Then, I go about adding a column to this table that says whether it matches the client or not, basically a 1 or a 0. Then, I filter out the zeros. Then I grab the MAX of the second column, that is my longest match.
This is looping in DAX.
So, explain again how your logic fits into this?
Steelwave is a Client Name and it matches Sterling up until 3 characters. The only match returned is Sterling, not Sterling and Steelwave because Steelwave is not an Ongoing Project name. So, you are left with Steelwave matching Sterling up until Ste. You can lower the thresholds but then you start running into issues with Dell, etc.
Anonymous - OK, this might actually be the one. I added a KillThreshold that was possible because I moved the cleaning to the end to maximum the length of the match. I got rid of the "Unit" match also. So, basically once I adjusted the cleaning of the special characters and spaces at the end, I was able to increase Threshold1 so that ABB is included but Unit is excluded. The only special exception is for Blue Cross and I'm not sure how else to handle that one quite honestly. Let me know if you find any obvious issues. Updated PBIX attached.
Fuzzy Column =
VAR __MatchWord = [Client Name]
VAR __CleanMatchThreshold = 4
VAR __KillThreshold = 3
VAR __FuzzyThreshold1 = .4
VAR __FuzzyThreshold2 = .8
VAR __WordSearchTable =
GENERATE(
'Ongoing Projects',
VAR __Word = MAXX(FILTER('Ongoing Projects',[Index]=EARLIER('Ongoing Projects'[Index])),[Ongoing Projects])
RETURN ADDCOLUMNS(GENERATESERIES(3,LEN(__Word),1),"Search",LEFT(__Word,[Value]),"Original",__Word)
)
VAR __Table =
FILTER(
ADDCOLUMNS(
__WordSearchTable,
"Match",SEARCH([Search],__MatchWord,,BLANK())
),
NOT(ISBLANK([Match]))
)
VAR __Max = MAXX(__Table,[Value])
VAR __Match = MAXX(FILTER(__Table,[Value]=__Max),[Search])
VAR __Proposed =
IF(
LEN(__Match)<=__CleanMatchThreshold,
SWITCH(TRUE(),
//__Clean2 = "ABB",__Clean2,
COUNTROWS(FILTER(__Table,[Value]=__Max))>1,BLANK(),
LEN(__Match) <= __KillThreshold,BLANK(),
LEN(__Match) = LEN(__MatchWord),__Match,
LEN(__Match)/LEN(__MatchWord)>__FuzzyThreshold1 && SEARCH(__Match,__MatchWord,,0)=1,__Match,
LEN(__Match)/LEN(__MatchWord)>__FuzzyThreshold2,__Match,
BLANK()
),
SWITCH(TRUE(),
__Match = "Blue Cross" || __Match = "Blue Cross ",__Match,
LEN(__Match)/LEN(__MatchWord)<__FuzzyThreshold2 && SEARCH(__Match,__MatchWord,,0)<>1,BLANK(),
__Match
)
)
VAR __Clean1 = IF(RIGHT(__Proposed,1)="(",LEFT(__Proposed,LEN(__Proposed)-1),__Proposed)
VAR __Clean2 = IF(RIGHT(__Clean1,1)=" ",LEFT(__Clean1,LEN(__Clean1)-1),__Clean1)
RETURN
__Clean2
Greg_DecklerI have attached the current list of client names for your reference. You could use this list in your working file if it is convenient.
Anonymous - So, yes, we can return the entire matching project, that we can do. I will see what I can do with that. The problem with Blue Cross is that the client names are so long that it ends up not meeting the threshold requirements for a match. I may be able to adjust those. So, for example
Blue Cross and Blue Shield of Arizona I
Matches 11 out of like 40 characters or .275% This makes it fall below thresholds set to filter out other stuff. The Ernst & Young entry for Blue Cross is even worse.
Anonymous - I will take a look.
Anonymous - OK, I updated this with a Client Name output version and Project output version in 2 additional columns. I checked the matches, don't see anything obviously amiss. Solved?
PBIX is attached.
Anonymous Glad we got there, was an interesting problem. There are still replies in this thread you haven't kudo'd... 🙂
This may be my record for longest thread before solution, 6 pages!!!
Greg_DecklerThank you Greg. You are truly a genius. I will go ahead and use this for my presentation today. This is really helpful. Thank you for all your time.
However I do not understand just one thing. Why was the exception given to Blue Cross?Even National Grid has two matches with client names similar to Blue cross. While this works for National Grid, why is it not working for Blue Cross?
(See -Ernst and YoungNATIONAL Grid and Ernst and YoungBLUE Cross under client names for which blue cross exception was added). While National grip is getting mapped to YoungNATIONAL Grid, Blue Cross should get mapped to YoungBLUE Cross since the client names are of similar nature or structure. Any thoughts on why this is happening? Then may be we can figure out to eliminate giving this exception as well and have a perfectly working formula without any exceptions 🙂
Let me know you thoughts. I will go ahead and accept this as a solution post your confirmation on this piece.Greg_Deckler Presented today with the exception but obviously I did not mentioned about the exception in the code. However I was thinking, instead of retunring only matched characters and then clean it, can we have the formula return the entire cell value from the Project list?
For example, instead of 'Casey' which has been matched, since 'Casey' is matched, the return value will be the entire cell value of 'Casey' i.e; Casey's General Store (CARES 2019) --> From the project list.
This should be similar for other matches.
Now I am aware of multiple names in project list (For example: Barclays has 3 variations in the project list). In such cases let the retun value be the first occurance od this match term. i.e; Barclays(VDA).
Sorry for trying to improvise on the initial requirement but could you suggest me a formula for the above?We can mark the response as a solution since I am satisfied with the results. (Except for my initial concern to avoid giving exception to 'Blue Cross'). You could also clarify why Blue Cross is behaving differently than 'National Grid'.
Thanl you so much for all the time Greg_Deckler I really appreciate. You are the saviour 🙂
Greg_Deckler For some wierd reason, I added data to my datadump today and after refresh I dont see Blue Cross and Blue Shield of Arizona I as well as The Ernst & Young entry for Blue Cross . I only see BLUE CROSS OF IDAHO CARE PLUS, INC in the data dump (excel). However when I filter in the data view of the BI desktop under list of projects, there are two entries for Blue cross. Idaho care plus and the Ernst and Young Entry. I am not sure what could be a reason for this. I dont see Blue Shield entry in my entire data set nor in the data view. Have you come across such odd things?
On the return value, we could either return the Project Name for the match or the Client Name for the Match. (either works - Better to know when to change what for convenience).
I am still trying to understand some part of the code. May be I can play around with the threshold values and get a match for Blue cross also for all kind of entries currently present. That is if you dont come up with one earlier than me 🙂
Please share the latest improvements you could make on the existing one to have better results. Hopefully the next post from you could be our very much needed near to perfect solution or even better - A perfect Solution 😉Greg_DecklerGenius!! Thank you so much for this. You have literally spared me a grave headache. Wish I could give you a dozen Kudos haha!! Peace!
Hi Greg, pleasure in greeting you... 2 queries:
1. Does the Microsoft Power BI Cookcbook (2nd Edition) have a Spanish version?
2. In the book, will you have cases to identify ranges of text and texts with momodines?
Example, if I have the value "AC23", "AJ80" and "KRZ600".
Number Since Until Caso1 AB01 AJ70 Caso2 A* J* Caso3 KRZ600 AC23 = Case1 and Case2
AJ80 = Caso2
KRZ600 = Caso3
Thanks and greetings...
