Child pages
  • Installing Python Packages
Skip to end of metadata
Go to start of metadata

Anaconda Environment

Background

Anaconda is a free and open-source distribution of the Python and R programming languages for scientific computing. The big difference between Conda and the pip package manager is in how the package dependencies are managed, which is a significant challenge for Python data science and the reason Conda exists.

(see webpage https://www.anaconda.com for details)

Existing Anaconda Environment

There are currently 6 Conda environments supported on HAL system

Environment NameLocationDescription
base/opt/apps/anaconda3Default Conda env with basic python packages.

deepspeed-v0.3.16

/opt/miniconda3/envs/deepspeed-v0.3.16

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

fastai-v0.1.18

/opt/miniconda3/envs/fastai-v0.1.18

fastai is a deep learning library that provides practitioners with high-level components that can quickly and easily provide state-of-the-art results in standard deep learning domains and provides researchers with low-level components that can be mixed and matched to build new approaches.
wmlce-v1.6.2/opt/anaconda3/envs/wmlce-v1.6.2Watson Machine Learning Community Edition is an IBM Cognitive Systems offering that is designed for the rapidly growing and quickly evolving AI category of deep learning.
wmlce-v1.7.0/opt/anaconda3/envs/wmlce-v1.7.0Watson Machine Learning Community Edition is an IBM Cognitive Systems offering that is designed for the rapidly growing and quickly evolving AI category of deep learning.
opence-v1.0.0

/opt/miniconda3/envs/opence-v1.0.0

Open-CE is a community-driven software distribution for machine learning that runs on standard Linux platforms with NVIDIA GPU technologies.
opence-v1.1.2/opt/miniconda3/envs/opence-v1.1.2Open-CE is a community-driven software distribution for machine learning that runs on standard Linux platforms with NVIDIA GPU technologies.
opence-v1.2.2/opt/miniconda3/envs/opence-v1.2.2Open-CE is a community-driven software distribution for machine learning that runs on standard Linux platforms with NVIDIA GPU technologies.
opence-v1.3.1/opt/miniconda3/envs/opence-v1.3.1Open-CE is a community-driven software distribution for machine learning that runs on standard Linux platforms with NVIDIA GPU technologies.
rapids

/opt/miniconda3/envs/rapids

The RAPIDS suite of software libraries, built on CUDA-X AI, gives you the freedom to execute end-to-end data science and analytics pipelines entirely on GPUs.

Create a New Env from Existing Envs

We recommend our users to create a new environment from one of our existing opence environment.

Create a New Env from Existing Env
conda create --name=<new_env> --clone=opence-v1.3.1

The new Conda environment will be located within $HOME/.conda/envs/<new_env>, then users can search and/or install python packages via Conda

Example: Search for a New Package
conda search r-tensorflow

Create Conda Environment from Scratch

Users can also create a new environment from scratch

Create a New Env from Existing Env
conda create --name=<new_env_name>

Important Note about Install package with pip

Some packages only support installing with "pip" and we allow users to install the package with "pip" within their own conda environment. However, install with "pip" is not always work since it could have a conflict with the Conda environment.


  • No labels