Forum Discussion

roshneematlani's avatar
roshneematlani
Icon for Microsoft Employee rankMicrosoft Employee
7 months ago
Solved

R Model training takes forever in Fabric Notebook

Hello,   I use GLM Model in R in databricks which usually takes 3-4 hrs for model training. I am trying to replicate same in the Fabric notebook in SparkR, and it is taking forver to run. Everytime...
data science
ml model
notebook
v-achippa's avatar
v-achippa
Icon for Community Support rankCommunity Support
7 months ago

Hi roshneematlani,

 

Currently there is no single official wiki page that explicitly calls out the limitation. But from the official Spark MLlib and SparkR documentation, we can see the set of supported GLM algorithms and parameters, and the bias reduced methods like Jeffreys/Firth are not part of that supported surface area. Please refer below documents:
https://spark.apache.org/docs/latest/ml-classification-regression

https://spark.apache.org/docs/latest/api/R/articles/sparkr-vignettes

 

Thanks and regards,

Anjan Kumar Chippa

 

v-achippa's avatar
v-achippa
Icon for Community Support rankCommunity Support
7 months ago

Hi @roshneematlani,

 

We wanted to kindly follow up to check if the solution I have provided for the issue worked? or let us know if you need any further assistance.

 

Thanks and regards,

Anjan Kumar Chippa