| Table of Contents | ||
|---|---|---|
|
Anaconda Environment
Background
Anaconda is a free and open-source distribution of the Python and R programming languages for scientific computing. The big difference between Conda and the pip package manager is in how the package dependencies are managed, which is a significant challenge for Python data science and the reason Conda exists.
(see webpage https://www.anaconda.comfor details)
Existing Anaconda Environment
There are currently 6 Conda environments supported on HAL system
| Environment Name | Location | Description |
|---|---|---|
| base | /opt/apps/anaconda3 | Default Conda env with basic python packages. |
deepspeed-v0.3.16 | /opt/miniconda3/envs/deepspeed-v0.3.16 | DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective. |
fastai-v0.1.18 | /opt/miniconda3/envs/fastai-v0.1.18 | fastai is a deep learning library that provides practitioners with high-level components that can quickly and easily provide state-of-the-art results in standard deep learning domains and provides researchers with low-level components that can be mixed and matched to build new approaches. |
| wmlce-v1.6.2 | /opt/anaconda3/envs/wmlce-v1.6.2 | Watson Machine Learning Community Edition is an IBM Cognitive Systems offering that is designed for the rapidly growing and quickly evolving AI category of deep learning. |
| wmlce-v1.7.0 | /opt/anaconda3/envs/wmlce-v1.7.0 | Watson Machine Learning Community Edition is an IBM Cognitive Systems offering that is designed for the rapidly growing and quickly evolving AI category of deep learning. |
| opence-v1.0.0 | /opt/miniconda3/envs/opence-v1.0.0 | Open-CE is a community-driven software distribution for machine learning that runs on standard Linux platforms with NVIDIA GPU technologies. |
| opence-v1.1.2 | /opt/miniconda3/envs/opence-v1.1.2 | Open-CE is a community-driven software distribution for machine learning that runs on standard Linux platforms with NVIDIA GPU technologies. |
| opence-v1.2.2 | /opt/miniconda3/envs/opence-v1.2.2 | Open-CE is a community-driven software distribution for machine learning that runs on standard Linux platforms with NVIDIA GPU technologies. |
| opence-v1.3.1 | /opt/miniconda3/envs/opence-v1.3.1 | Open-CE is a community-driven software distribution for machine learning that runs on standard Linux platforms with NVIDIA GPU technologies. |
| rapids | /opt/miniconda3/envs/rapids | The RAPIDS suite of software libraries, built on CUDA-X AI, gives you the freedom to execute end-to-end data science and analytics pipelines entirely on GPUs. |
Create a New Env from Existing Envs
We recommend our users to create a new environment from one of our existing opence environment.
| Code Block | ||||
|---|---|---|---|---|
| ||||
conda create --name=<new_env> --clone=opence-v1.3.1 |
The new Conda environment will be located within $HOME/.conda/envs/<new_env>, then users can search and/or install python packages via Conda
| Code Block | ||||
|---|---|---|---|---|
| ||||
conda search r-tensorflow |
Create Conda Environment from Scratch
Users can also create a new environment from scratch
| Code Block | ||||
|---|---|---|---|---|
| ||||
conda create --name=<new_env_name> |
Important Note about Install package with pip
Some packages only support installing with "pip" and we allow users to install the package with "pip" within their own conda environment. However, install with "pip" is not always work since it could have a conflict with the Conda environment.