Forum Discussion

Broeks's avatar
Broeks
Helper II
1 year ago

Using custom python library in notebook

I've create a whl file containing a python function and uploaded it as a custom library into my fabric environment. I've succesfully deployed the library and now I would expect the library to be available to import during a notebook session.

 

However, this is not the case. When I run the pip list, it's not shown in the list and I can also not call the library through the import function:


When I install the package from the whl file in the environment resource, the package does work. However I would expect to just add my whl file to the environment as a custom library and then be able to use the packages inside. Did I miss something or do i need the use the custom library in another way?

10 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi Broeks ,

    Can you please share some more detail information about this issue and sameple whl file ? They should help us clarify your scenario and test to troubleshoot.

    In addition, have you tried to use %pip list instead of native jupyter pip command to list the installed libraries?

    Regards,

    Xiaoxin Sheng

  • Hi Broeks ,

     

    Are you able to see your library in workspace settings of Spark?

     

     

    Thanks,

    Srisakthi

  • Thanks for your responses.
    To answer Srisakthi, yes the library is present:

    And to Anonymous, i've tried the %pip list and my package isn't present.

     

    Some additional info i can give is:

    - I've created the package my self, based on een small set of python files. However, since the package does work when i install it through a notebook cell, I''ve concluded that the package isn't the issue.


    This is the structure and one of the functions is the package:


    This is the code i've used for the creation of the package:

     

    from setuptools import find_packages, setup
    
    setup(
        name='idpdemo',
        version='0.0.1',
        description='demo_description',
        author='Broeks',
        packages=find_packages(include=['idp']),
        install_requires=[],  # Add your dependencies here
        extras_require={
            'dev': ['pytest']  # Test dependencies go here
        }
    )

    And I've tried to import the function through the following code:

    from idp import check_dataframe_for_duplicate_keys

    If you need additional information, let me known.

    • Anonymous's avatar
      Anonymous
      Not applicable

      HI Broeks,

      I'd like to suggest text with other whl format custom libraries to confirm if this issue only applies on your side.

      Regards,

      Xiaoxin Sheng

      • Broeks's avatar
        Broeks
        Helper II

        Yes, will do. I'm quitte busy these days, so a response might take a few days. But thanks for the help so far!