Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
9 years ago
Solved

R Limitations

I would like to know how i could handle the data export with R 

 

My dataset is more than 10millions line. 

I know that one export is limitted to 150k. 

So I write the following program to split the data into chuncks of data frame to be exported. 

I am getting only the first 150k and the second file is only 1 row.

Could you help me handle this?

 

numBreaks <- nrow(dataset)%/%150000 + 1
setwd("D:/test/")
for( i in seq(numBreaks)){
  filename = paste("test",i, sep="_")
  filename = paste(filename,"csv",sep=".")
  smallDAT <- dataset[((i-1)*150000+1): (min(nrow(dataset), i*150000)), ]
 write.table (smallDAT , file =filename, sep =",", row.names = FALSE)
}

 

  • Hi Anonymous,

    R visuals in Power BI Desktop has a few limitations:

    • Data size limitations – data used by the R visual for plotting is limited to 150,000 rows. If more than 150,000 rows are selected, only the top 150,000 rows are used and a message is displayed on the image.


    Considering this limitation which is the limit on all R-scripts at the moment on Power BI Desktop(For example, in your program above, the nrow(dataset) will always return 150000 when the dataset is more than 150,000 rows.), I don't think it is possible to break this limitation currently. :smileyhappy:

     

    Regards

3 Replies

  • v-ljerr-msft's avatar
    v-ljerr-msft
    Microsoft Employee

    Hi Anonymous,

    R visuals in Power BI Desktop has a few limitations:

    • Data size limitations – data used by the R visual for plotting is limited to 150,000 rows. If more than 150,000 rows are selected, only the top 150,000 rows are used and a message is displayed on the image.


    Considering this limitation which is the limit on all R-scripts at the moment on Power BI Desktop(For example, in your program above, the nrow(dataset) will always return 150000 when the dataset is more than 150,000 rows.), I don't think it is possible to break this limitation currently. :smileyhappy:

     

    Regards

    • Anonymous's avatar
      Anonymous
      Not applicable

      Thank you very much. 

      It is a really big problem for us now....

      I hope it will be solved soon. At least the export to csv should not have limit

      • BeardyGeorge's avatar
        BeardyGeorge
        Advocate I

        it's worth saying that you don't have to use Power BI to be your R environment, you can just use R-studio and you'll probably be fine.