topic Re: R Model training takes forever in Fabric Notebook in Data Science

R Model training takes forever in Fabric Notebook

roshneematlani — Fri, 16 Jan 2026 19:10:04 GMT

Hello,

I use GLM Model in R in databricks which usually takes 3-4 hrs for model training. I am trying to replicate same in the Fabric notebook in SparkR, and it is taking forver to run. Everytime session gets disconnected. Is it the known issue that sparkR in fabric does not support GLM Models or is v v slow? I tried increasing the capacity of Fabric Subscription but it is not helping in any way. Any suggestions on this?

Thanks

Re: R Model training takes forever in Fabric Notebook

v-achippa — Mon, 19 Jan 2026 08:44:26 GMT

Hi @roshneematlani,

Thank you for reaching out to Microsoft Fabric Community.

This is not a known issue with GLM support in Fabric. While SparkR is supported in fabric, GLM training via glm() in SparkR is driver bound and is not distributed. Increasing fabric capacity only adds executor resources, which does not improve SparkR GLM performance.

In fabric notebooks long running SparkR jobs can be very slow and may hit notebook session or driver limits, that causes the disconnections you are seeing. This behaviour differs from databricks, where SparkR clusters allow larger and more persistent drivers.

For large datasets or multi-hour training, the recommend way is to use Spark MLlib GLM (via sparklyr or PySpark), which is fully distributed and supported in fabric. SparkR is best for the smaller or exploratory workloads.

Thanks and regards,

Anjan Kumar Chippa

Re: R Model training takes forever in Fabric Notebook

roshneematlani — Mon, 19 Jan 2026 20:00:51 GMT

My current model is as follows-

fit <- glm( formula = y~ x1+ x2+ x3+ x4 + x5 + x6 + x7 + x8 + x9 + x10, data = train_df, family = binomial, weights = weights, method = "brglmFit", type = 'MPL_Jeffreys', a = 0.1, control = list( maxit = 5000, trace = TRUE, slowit = 0.1 ) )

and based on my research- using
method = "brglmFit"
type = "MPL_Jeffreys"
in fabric sparkR is very slow and equivalent is NOT available in Spark MLlib.

Any other alternative for fabric?

Re: R Model training takes forever in Fabric Notebook

v-achippa — Thu, 22 Jan 2026 05:18:34 GMT

Hi @roshneematlani,

Yes you are correct, brglmFit with MPL_Jeffreys has no equivalent in Spark MLlib, and that is why the workload performs poorly in fabric sparkR. These models are strictly single node and run on the spark driver, because of that long running training disconnects in fabric.

There is no equivalent or better alternative inside fabric today.

If jeffreys prior bias reduction is required, the recommended approach is to train the model outside fabric using native R and then deploy the trained coefficients for scoring inside fabric.

Thanks and regards,

Anjan Kumar Chippa

Re: R Model training takes forever in Fabric Notebook

v-achippa — Tue, 27 Jan 2026 12:58:45 GMT

Hi @roshneematlani,

As we haven’t heard back from you, we wanted to kindly follow up to check if the solution I have provided for the issue worked? or let us know if you need any further assistance.

Thanks and regards,

Anjan Kumar Chippa

Re: R Model training takes forever in Fabric Notebook

roshneematlani — Tue, 27 Jan 2026 16:36:10 GMT

Do we have any official wiki or any other resource which states this limitation?

Thanks

Roshnee

Re: R Model training takes forever in Fabric Notebook

v-achippa — Wed, 28 Jan 2026 07:52:12 GMT

Hi @roshneematlani,

Currently there is no single official wiki page that explicitly calls out the limitation. But from the official Spark MLlib and SparkR documentation, we can see the set of supported GLM algorithms and parameters, and the bias reduced methods like Jeffreys/Firth are not part of that supported surface area. Please refer below documents:
https://spark.apache.org/docs/latest/ml-classification-regression

https://spark.apache.org/docs/latest/api/R/articles/sparkr-vignettes

Thanks and regards,

Anjan Kumar Chippa

Re: R Model training takes forever in Fabric Notebook

v-achippa — Mon, 02 Feb 2026 15:43:21 GMT

Hi @roshneematlani,

We wanted to kindly follow up to check if the solution I have provided for the issue worked? or let us know if you need any further assistance.

Thanks and regards,

Anjan Kumar Chippa

Re: R Model training takes forever in Fabric Notebook

Ravella_Vikhil — Mon, 16 Feb 2026 14:27:48 GMT

```
Train brglmFit model locally (CPU/GPU)
```
```
Save model coefficients
```

Load them into:

```
Django backend
```
```
Or inference-only service
```

Fabric is then used for:

```
Data ingestion
```
```
Feature prep
```
```
Analytics
```