Forum Discussion

maganstephens's avatar
maganstephens
New Member
9 years ago

word frequency

I have a bunch of text that I'm interested in seeing if there is a trend in the words used. Word Cloud is great but I can't figure out how to count the words and the frequency of each word after Word Cloud removes the stop words. Is there a way to do this?

Thanks,
Magan

3 Replies

  • Phil_Seamark's avatar
    Phil_Seamark
    Microsoft Employee
    9 years ago

    Hi maganstephens,

     

    I think this is even better.  Use the following R-Script in the Query Editor to add a column that will have the count of words in another column that contains the wordcount from another column

     

    I used a basic table and the column that this counts is called Words

     

     

    # 'dataset' holds the input data for this script
dataset = within(dataset , {WordCount= sapply(gregexpr("\\b\\W+\\b", Words, perl=TRUE), function(x) sum(x>0) ) + 1 })
output <- dataset

  • Phil_Seamark's avatar
    Phil_Seamark
    Microsoft Employee
    9 years ago

    Hi maganstephens,

     

    You can use R to count words

     

    I did a google for some scripts to count words using R and had success with a simple example.

     

    This example uses a table with 1 row called Column1 and plotted the correct number of words to the screen.

     

    str1 <- dataset$Column1
str2 <- gsub(' {2,}',' ',str1)
plot(length(strsplit(str2,' ')[[1]]))