wbob's avatar
wbob
Advocate I
10 years ago
Solved

Wordcloud with r

Although there is a custom visualisation for Wordcloud now, I did this one with r in Power BI Desktop as you get a greater degree of control over it:

 

 

library(tm)
library(wordcloud)

words <- Corpus(VectorSource(dataset))
words <- tm_map(words, stripWhitespace)
words <- tm_map(words, content_transformer(tolower))
words <- tm_map(words, removeNumbers)
words <- tm_map(words, removePunctuation)
words <- tm_map(words, removeWords, stopwords("english"))
words <- tm_map(words, stemDocument)
wordcloud(words, scale=c(5,0.5), max.words=50, random.order=FALSE, rot.per=0.35, use.r.layout=FALSE, colors=brewer.pal(8, "Dark2"))

 

I have attached a sample .pbix file with the text from Wikipedia's 'big data' article and the R visualisation.

  • When I tried the original code, it worked with older versions of R, 3.2.3 but not newer versions of R, 3.3.1 and 3.4.2. The Corpus construction was returning only numbers which were then stripped out by the rest of the code causing problems. So, I hacked together a variation of the original code based upon comments and have a working version here:

     

    require(tm)
    require(wordcloud)
    require(RColorBrewer)
    
    datain = as.data.frame(table(as.character(dataset[,1])))
    words <- Corpus(VectorSource(dataset$text))
    
    words <- tm_map(words, stripWhitespace)
    words <- tm_map(words, content_transformer(tolower))
    words <- tm_map(words, removeNumbers)
    words <- tm_map(words, removePunctuation)
    words <- tm_map(words, removeWords, stopwords("english"))
    words <- tm_map(words, stemDocument)
    
    wordcloud(words, scale=c(5,0.75), max.words=50, random.order=FALSE, rot.per=0.35, use.r.layout=FALSE, colors=brewer.pal(8, "Dark2"))

    The main change is the construction of the Corpus, VectorSource(dataset) becomes VectorSource(dataset$text)

     

    This is based upon the original PBIX file included in the original post.

13 Replies

  • Greg_Deckler's avatar
    Greg_Deckler
    Community Champion

    When I tried the original code, it worked with older versions of R, 3.2.3 but not newer versions of R, 3.3.1 and 3.4.2. The Corpus construction was returning only numbers which were then stripped out by the rest of the code causing problems. So, I hacked together a variation of the original code based upon comments and have a working version here:

     

    require(tm)
    require(wordcloud)
    require(RColorBrewer)
    
    datain = as.data.frame(table(as.character(dataset[,1])))
    words <- Corpus(VectorSource(dataset$text))
    
    words <- tm_map(words, stripWhitespace)
    words <- tm_map(words, content_transformer(tolower))
    words <- tm_map(words, removeNumbers)
    words <- tm_map(words, removePunctuation)
    words <- tm_map(words, removeWords, stopwords("english"))
    words <- tm_map(words, stemDocument)
    
    wordcloud(words, scale=c(5,0.75), max.words=50, random.order=FALSE, rot.per=0.35, use.r.layout=FALSE, colors=brewer.pal(8, "Dark2"))

    The main change is the construction of the Corpus, VectorSource(dataset) becomes VectorSource(dataset$text)

     

    This is based upon the original PBIX file included in the original post.

    • boefraty's avatar
      boefraty
      Microsoft Employee

      Thanks a lot, we will review the PBIX. Excpecially because the R-engine is going to be upgraded in service soon

  • Getting an error when trying to render in PowerBI.com.  Works fine in PowerBI Desktop...

    ERROR:

    [Monitor] Loaded provider: [WindowsLibOsProvider] [Monitor] Loaded provider: [SocketStreamProvider] [Monitor] Loaded provider: [DnsStreamProvider] [Monitor] Loaded provider: [HttpServerStreamProvider] [Monitor] Loaded provider: [ClockStreamProvider] [Monitor] Loaded provider: [ResourceManager] [Monitor] Loaded Security Monitor Profile: [C:\Sandboxing\Sandboxes\5a4800e1-dd61-41e3-9bce-3fb922b5f659\Profile\0.dbconfig] [Monitor] Writable folder: [file:///Users] => [C:\Sandboxing\Sandboxes\5a4800e1-dd61-41e3-9bce-3fb922b5f659\Users] [Monitor] Writable folder: [file:///Results] => [C:\Sandboxing\Sandboxes\5a4800e1-dd61-41e3-9bce-3fb922b5f659\Results] [Monitor] Writable folder: [file:///Work] => [C:\Sandboxing\Sandboxes\5a4800e1-dd61-41e3-9bce-3fb922b5f659\Work] [Monitor] R/O folder: [file:///ThirdParty] => [C:\WFRoot\SBRole.0\Fabric\work\Applications\ASAzureApp_App0\temp\R_Root_13.0.1605.329] [Monitor] R/O folder: [file:///Script] => [C:\Sandboxing\Sandboxes\5a4800e1-dd61-41e3-9bce-3fb922b5f659\Script] [Monitor] R/O folder: [file:///InputData] => [C:\Sandboxing\Sandboxes\5a4800e1-dd61-41e3-9bce-3fb922b5f659\InputData] [Monitor] Scratch folder: [C:\Sandboxing\Sandboxes\5a4800e1-dd61-41e3-9bce-3fb922b5f659\Scratch] [Monitor:WARNING] Cannot find dependency: "Windows.0.0.0.0.0" x64 [Monitor:WARNING] Unable to resolve package dependencies: 0x80070490 [Monitor:ERROR] Unable to create application sandbox environment; the most likely cause of this error is a missing package dependency. Please confirm that all application dependencies are present. (Status=0x80070490) [Monitor:ERROR] Failed to launch application: 0x80070490 [Monitor:WARNING] Failed to retrieve application's exit code! [Monitor] Done.
    Please try again later or contact support. If you contact support, please provide these details.

    • boefraty's avatar
      boefraty
      Microsoft Employee

      Dear erikskov,

      Thank you very much for the input and sorry for late response.

      Can I please get PBIX with data and R code, which is not working for you in server? It worked for me on toy data sample.

       

      Best Regards,

      B. ([email protected])

       

       

       

       

  • mike_honey's avatar
    mike_honey
    Memorable Member

    wbob - this seems an incomplete sample and perhaps why it is generating confusion.  I dont think that code alone can be used in PBI Desktop without a source datatset.

     

    Can you please post your PBIX file?

  • kavasimihaly's avatar
    kavasimihaly
    Frequent Visitor

    Hi, Thanks for sharing.

     

    I have a problem with special charaters braking the visualisation.

     

    Can you help me what to do with it.

     

    The R code is working when no special charaters present.

     

    This is the Error Message i get:

    Error Message:

    R script error.
    Error in type.convert(data[[i]], as.is = as.is[i], dec = dec, numerals = numerals, :
    invalid input '@MSEmpresasLatam @DiegoBek exactamente! Es la era de la inteligencia #AzureML #SQLEnterprise #PowerBI #IoT 😍' in 'utf8towcs'
    Calls: read.csv -> read.table -> type.convert
    Execution halted

     

     

    This is the R code i have:

    library(tm)
    library(SnowballC)
    library(wordcloud)
    dataset <-Corpus(VectorSource(dataset))
    dataset <- tm_map(dataset, removePunctuation)
    dataset <-tm_map(dataset, removeWords, c('powerbi','PowerBI',stopwords('english')))
    dataset <-tm_map(dataset, stemDocument)
    wordcloud(dataset, max.words = 100, random.order = FALSE)

     

    Thanks in advance if anyone could help.

     

    Best Regards,

    Kávási Mihály

    • mike_honey's avatar
      mike_honey
      Memorable Member

      Hi Kávási,

       

      It looks you are using the R Word Cloud, not the Power BI Custom Visual?  If that is the case then I suggest you re-post this e.g. to stackoverflow.com, tagged for R.

       

      • kavasimihaly's avatar
        kavasimihaly
        Frequent Visitor

        Hi,

         

        the reason why i posted here because this is the R script showcase forum and this error is related to the R integration of PowerBI.

         

        Best regards, 

        Kávási Mihály ( in English it is Mihály Kávási correctly :) will change from now)

  • Hi,

     

    Nice post :-)

    Do you know how to make a word cloud out of a columns (the entire content) for example i have a colum with company names and number of cases?

     

    The problem is that by altering the Dataset I cannot use the code again in Power BI ( I had to write colClasses = c("character", "numeric"),  to tell R that my dataset has 2 type of columns.

     

    Thanks!

     

    > `dataset` = read.csv('C:/Users/PatricioLobos/AppData/Local/Radio/REditorWrapper_8fe60428-ecc0-478f-ab77-56803f9407f4/input_df_227d85ab-962e-4701-b1ac-ff6e53817c56.csv', colClasses = c("character", "numeric"), check.names = FALSE, encoding = "UTF-8", blank.lines.skip = FALSE);
    + # Original Script. Please update your script content here and once completed copy below section back to the original editing window #
    >require(wordcloud)
    > require(RColorBrewer)
    > pal2 <- brewer.pal(8, 'Dark2')
    > plot(wordcloud(dataset$Kundenavn, dataset$`Nr. Cases`, scale = c(8, .4), min.freq = 5, max.words = Inf, random.order = FALSE, rot.per = .15, colors = pal2))

    • boefraty's avatar
      boefraty
      Microsoft Employee
      require(wordcloud)
      require(RColorBrewer)
      
      datain = as.data.frame(table(as.character(dataset[,2])))
      
      pal2 <- brewer.pal(4,"Dark2")
      
      wordcloud(datain[,1],datain$Freq, 
                scale=c(4,.3),
                min.freq=1, max.words=Inf, 
                random.order=FALSE, 
                rot.per=.15, 
                colors=pal2)

       The dataset is 2 columns: "ID", "Term" . If you have "Frequency" instead of "ID" no need to call "table" operator  

       

       

  • ToveHjelm's avatar
    ToveHjelm
    Regular Visitor

    I love the idea with a word cloud that you have more control over. However, when I open the pbix-file to try the visual I get the message "Can't display visuals", with the following details:

    Error Message:

    R script error.

    Loading required package: NLP

    Loading required package: methods

    Loading required package: RColorBrewer

    Error in strwidth(words[i], cex = size[i], ...) : invalid 'cex' value

    Calls: wordcloud -> strwidth

    In addition: Warning messages:

    1: In max(freq) : no non-missing arguments to max; returning -Inf

    2: In max(freq) : no non-missing arguments to max; returning -Inf

    Execution halted

     

    I have the packages installed. Any ideas?

    • Datamijn's avatar
      Datamijn
      Helper I

      I got the same error when working on my own word cloud. I solved it by adding the 

      'require("package")' for all the packages used. In my case it worked after adding the require call, but the end of the script is slightly different than the example here.

       

      require("tm")
      require("SnowballC")
      require("wordcloud")
      require("RColorBrewer")

       

      library(tm)
      library(SnowballC)
      library(wordcloud)
      library(RColorBrewer)

       

      #Data
      docs <- Corpus(VectorSource(dataset$Column1))

      # Convert the text to lower case
      docs <- tm_map(docs, content_transformer(tolower))
      # Remove numbers
      docs <- tm_map(docs, removeNumbers)
      # Remove english common stopwords
      #docs <- tm_map(docs, removeWords, stopwords("dutch"))
      # Remove your own stop word
      # specify your stopwords as a character vector
      docs <- tm_map(docs, removeWords, c("het", "met", "ons", "dit", "hem", "als", "dat", "heb"))
      # Remove punctuations
      docs <- tm_map(docs, removePunctuation)
      # Eliminate extra white spaces
      #docs <- tm_map(docs, stripWhitespace)
      # Text stemming
      #docs <- tm_map(docs, stemDocument)

      #Build a term-document matrix
      dtm <- TermDocumentMatrix(docs)
      m <- as.matrix(dtm)
      v <- sort(rowSums(m),decreasing=TRUE)
      d <- data.frame(word = names(v),freq=v)
      head(d, 10)

      set.seed(1234)
      wordcloud(words = d$word, freq = d$freq, min.freq = 1,
      max.words=200, random.order=FALSE, rot.per=0.35,
      colors=brewer.pal(8, "Dark2"))

       

       

      • shadho's avatar
        shadho
        Regular Visitor

        Hi, 

        I pretty much did the same exact steps to generate a word cloud visual in R , however i'm still getting  the same error.

        The script runs fine on my machine.

         

        Here is the code snippet

         

        # Paste or type your script code here:
        require("tm")
        require("SnowballC")
        require("wordcloud")
        require("wordcloud2")
        require("RColorBrewer")

        library(tm)
        library(SnowballC)
        library(wordcloud)
        library(RColorBrewer)
        library(wordcloud2)

        #Data
        docs <- Corpus(VectorSource(dataset$UserFeedback))

        ##Data Cleaning and Wrangling
        docs <- tm_map(docs, PlainTextDocument)
        docs <- tm_map(docs,tolower)
        docs <- tm_map(docs,removeNumbers)
        docs <- tm_map(docs,removeWords,c(stopwords("english"),"expert","just","asdfasdf","asdf","session","slide","also","slides","asked","test","deck","around","took","look","might","make","many","give","keep","adding","going","go","making","really","job","ones","even","see","made","bit"))
        docs <- tm_map(docs,removePunctuation)
        docs <- tm_map(docs,stripWhitespace)
        docs <- tm_map(docs,stemDocument)

        tdm<-TermDocumentMatrix(docs)
        m <-as.matrix(tdm)
        v <-sort(rowSums(m),decreasing = TRUE)
        Docs2 <-data.frame(word=names(v),freq =v)
        head(Docs2, 10)
        wordcloud2(Docs2, color = "random-dark",size = 0.5, shape = "circle", backgroundColor = "black")
         
        Any thoughts what  could be the issue?