Unable to import SMOTE

[ericCheungBI]

Hi I'm notebooks and I'm trying to run 

!pip install imbalanced-learn

import imblearn

from imblearn.over_sampling import SMOTE

But I keep getting this error.
ImportError: cannot import name 'UnsetMetadataPassedError' from 'sklearn.exceptions' (/home/trusted-service-user/cluster-env/trident_env/lib/python3.11/site-packages/sklearn/exceptions.py)

I looked around it and sounds like I need to update my python version. This doesn't really make sense to me because I used this notebook in November 2024 and it ran perfectly fine. Why would I need to update anything?

[v-vpabbu]

Hi @ericCheungBI,

Thank you for reaching out to Microsoft Fabric Community Forum.

Even though the notebook worked before, some libraries may have been updated since then. These updates could introduce breaking changes or incompatibilities, which is often why such errors appear even on previously working code.

Try updating the library to the latest version and let me know if you're still having issues.

Did I answer your question? Mark my post as a solution, this will help others!
If my response(s) assisted you in any way, don't forget to give "Kudos"

Regards,
Vinay Pabbu

[ericCheungBI]

Hey Vinay, I tried doing that, but that requires updating SKLearn to 1.6.0. After updating, it says to restart the kernel to use the updated packages. Is there a way to restart?

[v-vpabbu]

Hi @ericCheungBI,

To restart the kernel, follow these steps:

After connecting to the session, you will see the option to restart the kernel.

For reference:

Regards,

Vinay Pabbu

[ericCheungBI]

Do I need a choose a setting to make that show up?

[v-vpabbu]

Hi @ericCheungBI,

By default, it will be available in Python, but it will not be available in PySpark.

Regards,

Vinay Pabbu

[ericCheungBI · Accepted solution]

That solution isn't quite what I'm looking for since this notebook is part of a pipeline that needs to automatically run and already uses pyspark.

The solution I did find is to use %pip install scikit-learn==1.3.0. I don't have a python background, so I wasn't aware that there's a difference between !pip and %pip, but the microsoft documentation suggests %pip as it allows you to use a specific library for the current session.  Additionally since this is a pipeline, you need to include a bool parameter parameter_inlineInstallationEnabled = true to the notebook block.

See Python inline installation from the following docs.

Manage Apache Spark libraries - Microsoft Fabric | Microsoft Learn