Forum Discussion

masplin's avatar
masplin
Icon for Impactful Individual rankImpactful Individual
5 years ago
Solved

Struggling with TOPN

I am trying to classify companies based on the frequency of specific words in their description which each work being associated with a differnet sector.      Here is a description "We provide serv...
  • d_gosbell's avatar
    5 years ago

    Yes, it was a bit hard to tell if it was working with just the one row of test data. I think the following is closer, but potentially you could still end up with multiple sectors with the same score as I'm not sure how you would want to break the ties.

     

    Column = CONCATENATEX(
        var _topn =
        TOPN(1, ALL(Testing_Keyword[Sector]),
            var _currentSector = Testing_Keyword[Sector]
            var _sectorKeywords = CALCULATETABLE(GROUPBY(Testing_Keyword, Testing_Keyword[Weight], Testing_Keyword[Keyword]), TREATAS( {_currentSector}, Testing_Keyword[Sector] ))
            var _score = SUMX(
                _sectorKeywords,
                Testing_Keyword[Weight]  * 
                (LEN(LOWER(CRO_Company[Description])) - LEN(SUBSTITUTE(LOWER(CRO_Company[Description]),LOWER(Testing_Keyword[Keyword]),""))) /
                LEN(Testing_Keyword[Keyword])
                )
            return _score
        )
        // if the _topn variable contains all the rows from the keyword table this probably
        // means that none of the keywords matched so they all scored 0 so we should 
        // filter them all out
        var _condition = if(COUNTROWS(all(Testing_Keyword[Sector])) = COUNTROWS(_topn),FALSE(), True())
        return filter( _topn, _condition)
    ,[Sector]
    ,",")

     

    While I was testing this I also built the following column so I could see the weighted score per sector, this may be helpful if you want to do further debugging yourself.

     

    Column 2 = 
    CONCATENATEX( ALL(Testing_Keyword[Sector]),
    var _currentSector = Testing_Keyword[Sector]
    var _sectorKeywords = CALCULATETABLE(GROUPBY(Testing_Keyword, Testing_Keyword[Weight], Testing_Keyword[Keyword]), TREATAS( {_currentSector}, Testing_Keyword[Sector] ))
    var _score = SUMX(
            _sectorKeywords,
            Testing_Keyword[Weight]  * 
            (LEN(LOWER(CRO_Company[Description])) - LEN(SUBSTITUTE(LOWER(CRO_Company[Description]),LOWER(Testing_Keyword[Keyword]),""))) /
            LEN(Testing_Keyword[Keyword])
            )
            return _currentSector & " (" & _score & ") "
    )