Forum Discussion
Unable to import SMOTE
That solution isn't quite what I'm looking for since this notebook is part of a pipeline that needs to automatically run and already uses pyspark.
The solution I did find is to use %pip install scikit-learn==1.3.0. I don't have a python background, so I wasn't aware that there's a difference between !pip and %pip, but the microsoft documentation suggests %pip as it allows you to use a specific library for the current session. Additionally since this is a pipeline, you need to include a bool parameter parameter_inlineInstallationEnabled = true to the notebook block.
See Python inline installation from the following docs.
Manage Apache Spark libraries - Microsoft Fabric | Microsoft Learn
Hi ericCheungBI,
By default, it will be available in Python, but it will not be available in PySpark.
Regards,
Vinay Pabbu
That solution isn't quite what I'm looking for since this notebook is part of a pipeline that needs to automatically run and already uses pyspark.
The solution I did find is to use %pip install scikit-learn==1.3.0. I don't have a python background, so I wasn't aware that there's a difference between !pip and %pip, but the microsoft documentation suggests %pip as it allows you to use a specific library for the current session. Additionally since this is a pipeline, you need to include a bool parameter parameter_inlineInstallationEnabled = true to the notebook block.
See Python inline installation from the following docs.
Manage Apache Spark libraries - Microsoft Fabric | Microsoft Learn