topic Re: Data Fabric, R language -Custom libraries installation failed in Data Science

Data Fabric, R language -Custom libraries installation failed

ming_ — Wed, 04 Sep 2024 08:46:03 GMT

I'm trying to upload the custom library of R named tsintermittent using tar.gz file. But it keep showing that it is failed to publish. Below is the error message return when I'm trying to upload the library package.

So, alternatively, I'm using the option for inline installation, it able to install library in the notebook. However, when I use the notebook in the pipeline, it returns to error message of

I would like to seek help in any other ways to install the R library in custom environment or how to enable the setting of inline installation in pipeline when running R notebook?

Appreciate the helps and inputs!

Re: Data Fabric, R language -Custom libraries installation failed

Anonymous — Thu, 05 Sep 2024 07:10:14 GMT

Hi ming_

The commands of managing R libraries are disabled when running pipeline jobs and there is no option to enable the inline installation of R libraries in pipeline currently. We need to select a custom environment for running the notebook in a data pipeline.

When installing the custom R library "tsintermittent" to a custom environment, I could reproduce the same error when trying to publish the environment. I found the following warning message in the error log. It may be a dependency error. You can look for similar errors in your log.

By far, based on my test, installing only 'MAPA' library is not enough as it lacks other dependencies. I'm looking for and installing all the dependencies. I will keep you updated when I have any progress. You may give it a try at your end. Hope this would be helpful.

Best Regards,
Jing
If this post helps, please Accept it as Solution to help other members find it. Appreciate your Kudos!

Re: Data Fabric, R language -Custom libraries installation failed

ming_ — Thu, 05 Sep 2024 08:08:32 GMT

Hi Anonymous ,
Thanks for your valuable input. Previously I did try to install a few dependencies packages and it return the same error when upload the related packages (Permission denied). I will try to track all the dependencies packages and install them see if it will be success.

Re: Data Fabric, R language -Custom libraries installation failed

Anonymous — Thu, 05 Sep 2024 09:49:44 GMT

Hi ming_

It's a long dependency list, totally 19 libraries... After installing all of them, the publish succeeded at my end finally!

This is my list:

If you need more custom libraries, the dependency list might be longer. It is time consuming but at least hopeful now. Look forward to your good news!

Best Regards,
Jing
If this post helps, please Accept it as Solution to help other members find it. Appreciate your Kudos!

Re: Data Fabric, R language -Custom libraries installation failed

ming_ — Fri, 06 Sep 2024 13:24:23 GMT

Hi Anonymous ,

Thanks for your valuable solution. I manage to identify all the dependecies library and install it in the custom environment.

I faced some issues to upload a few libraries even all the dependencies library has installed, but I created the new environment and re-install all the libraries again in sequence, now I could run the pipeline now.

Thanks a lot !

I have 1 follow up question from here, is there a way to share these existing library across different workspace rather than manually upload all the libraries to different workspace?

Re: Data Fabric, R language -Custom libraries installation failed

Anonymous — Mon, 09 Sep 2024 02:47:34 GMT

Hi ming_

* I faced some issues to upload a few libraries even all the dependencies library has installed.

A: I faced the same issue. My experience was to delete all existing custom libraries from the environment, publish the environment. Then upload all custom libraries, publish the environment again.

* I have 1 follow up question from here, is there a way to share these existing library across different workspace rather than manually upload all the libraries to different workspace?

A: To my knowledge, there is currently no way to share these custom libraries across different workspaces. Each environment is isolated. You have to upload all libraries to an environment in a different workspace.

Best Regards,
Jing

Re: Data Fabric, R language -Custom libraries installation failed

ming_ — Mon, 09 Sep 2024 02:51:01 GMT

Hi Anonymous ,

Appreciate and thanks for your help.