Python Module Not Found Error When Importing Pandas: 'no module named 'fsspec_wrapper.trident.core'

[tarainfotech]

Hi,

I have numerous notebooks in which I utilize Python, and specifically the Pandas library for data transformations. Today I started to receive errors on the runs of multiple notebooks across different workspaces when running the line of code that imports the pandas module. Message: 'from fsspec_wrapper.trident.core import OneLakeFileSystem -> ModuleNotFoundError: No module named 'fsspec_wrapper.trident.core'. This error specifically occurs only when the notebook environment is a custom environment rather than "Workspace Default". These notebooks have ran for months with no issue and no manual changes were made to the custom environment, and just started to throw this error today. I tried adding the newest fsspec library (2/1/2025 version) from pypl using 'public library' into my custom environment, and importing fsspec prior to pandas in the notebook.. still got the same error. I have not been able to find a solution, [other than for all notebooks throwing this error, changing the environment from a custom one to 'Workspace default'- this solves the issue]. But sometimes the packages I have to use in a notebook are not included in the default workspace libraries, and I will have to use a custom environment and pandas- how can I solve this issue? Thanks so much!

[dtacct]

This issue is back, under the new Fabric Runtime 1.3 (Spark 3.5 and Delta 3.2).  Upgrading the environment temporarily fixed the issue. I left it alone for a few days, and the problem is back. Super frustrating. 

[dtacct]

This time we manually added fsspec into the Public Libraries, using the latest version of fsspec. Prior to that, we relied on the fsspec version that was in the Built-in Libraries. This fixed the problem.

Again, nothing from Microsoft on why that built-in library failed, and I've seen recent posts from others that other libraries than this one were failing (one post referred to azure.core or something like that).

[Mrizzo21]

Not sure if you have resolved your issue, but I had the exact same problem today in my Fabric notebooks that are utilizing Python that have run without issue until now. I also run a custom environment and found that one of my public libraries had a update available; my 'pyarrrow' was originally installed as 19.0.0.  I see now that 19.0.1 was available. I updated the library, re-published it and my notebook ran without issue.

[dtacct]

We tested out upgrading to Fabric Runtime 1.3 (Spark 3.5 and Delta 3.2) for our custom environment, and this appears to have resolved the issue. That doesn't explain why it's breaking for many people, but seems to have gotten us over the issue.

I didn't get a chance to test the recommendation from Pradeep_O_S, but that also sounded like a good suggestion to try if you're not able to just upgrade the entire runtime.

[Pradeep_O_S]

I experienced the same problem recently. To fix it, simply install the 'fsspec' package from PyPI within your custom environment. This resolved the issue for me.

[dtacct]

Thanks for posting this. We're having this exact problem too around the timeline you mentioned.

[V-yubandi-msft]

Hi @tarainfotech ,

Thank you for reaching out to the Microsoft Fabric Community. The error you are encountering usually indicates one of two issues

1. The pandas library is not installed in your Python environment.
2. Your Python script is running in an environment where pandas is not available.

To troubleshoot and resolve this issue, please follow these steps.

1. Install pandas using pip. If you are using Python 3, you might need to use pip3.
2. Verify that pandas is correctly installed by checking its version and use virtual environments to manage project-specific dependencies.
3. Ensure that pandas is in your Python path. If problems persist, try uninstalling and reinstalling pandas.
4. Install pandas’ dependencies separately if needed. Also, upgrade pip or setuptools to avoid installation issues.
5. If you have multiple Python versions, ensure you are using the correct one. On Windows, verify that the Python Scripts directory is in your system's PATH.

By following these steps, you should be able to resolve the error and ensure that your Python environment is properly set up to use pandas.

If this post is helpful, kindly Accept it as the solution to assist other members in finding it more quickly.

[mark_carlisle]

Clearly AI generated response, given the reference to Windows.

We are also impacted by this a real response to the issue would be appreciated.

[V-yubandi-msft]

Hello @mark_carlisle ,

@mark_carlisle , I want to clarify that this is not an AI-generated response.

My initial response was too generic, focusing on standard Python troubleshooting instead of the specific Microsoft Fabric Notebook environment. I understand how this may have seemed unrelated to the issue at hand.

The response was based on general best practices for resolving Python module-related errors, using resources like PyTutorial. However, I now realize that Fabric Notebooks have a managed runtime, and typical Python troubleshooting steps may not apply here. I apologize for the misunderstanding.

Thank You.

[tarainfotech]

Unfortunately this suggestion does not solve my issue. I am using a Fabric Notebook item, such as one you can add in a Fabric Workspace in the data engineering experience (for example- a notebook that is created by opening a fabric workspace, selecting 'New Item', then selecting 'Notebook').

When I run '!pip list' in the notebook code block using a standard environment (displays Environment: 'Workspace default' at the top of the notebook) it prints the following in the list of installed modules: 'pandas 2.0.3, fsspec 2023.10.0, fsspec_wrapper 0.1.15'.

I did note today that in a Fabric workspace when I navigate to: 'new item-> environment-> built in libraries -> python', it lists the following as 'built in': pandas version 2.0.3, fsspec 2023.10.0, fsspec_wrapper version 0.1.12. This leads me to wonder if a newly introduced version mismatch on the Fabric side is the cause of the issue. Was there a recent Fabric update that could cause this? As I did not make any manual changes to the notebook or environment... the code that imports the  modules stopped working one morning after previously having no issues with months of runs.

I tried to see if I could force add the updated wrapper version into the fabric 'environment' item from Pypl or Conda.. . however it shows 'Library is not found' when I try to search for fsspec_wrapper in the offerings. I am unable to install new modules directly into a fabric notebook such as inside a code block, as I get the message that you need to have sudo credentials to do this. Thus I am still unable to solve this issue. I hope these additional details are helpful. Any assistance is greatly appreciated!

[V-yubandi-msft]

Hi @tarainfotech ,

Apologies for the misunderstanding, @tarainfotech . I initially focused on general Python environments, but your issue is specifically with Microsoft Fabric Notebooks.

It appears the problem might stem from a version mismatch between the built-in Fabric libraries and your custom environment. As Fabric frequently updates its libraries, this can sometimes cause compatibility issues with custom environments.

Please compare the versions of your custom environment and the Workspace Default environment. If they differ, try aligning them.

Some users have successfully resolved this by installing the fsspec package from PyPI within their custom environment (as suggested by @Pradeep_O_S ). However, since you have already tried this approach, it seems it didn't resolve the issue in your case.

@dtacct mentioned that upgrading to Fabric Runtime 1.3 (Spark 3.5 and Delta 3.2) resolved the issue in their custom environment. Could you try this and see if it helps?

@dtacct Thank you for your inputs.

Please let us know if the issue persists after these steps. I hope this helps.

Thank You.

[dtacct]

I think this answer misses the point and is a stock response. The original message said that the user was running a notebook with pandas for months, so the library was clearly installed. We have the same problem they mentioned. Something has probably broken on the Microsoft Fabric side.