| Info | ||
|---|---|---|
| ||
See Getting started with Open Cognitive Environment (OpenCE, former WMLCE) for the latest software stack. |
| Table of Contents |
|---|
IBM Watson Machine Learning Community Edition (WMLCE-1.7.0, WMLCE-1.6.
...
2)
WMLCE is an enterprise software distribution that combines popular open-source deep learning frameworks, efficient AI development tools, and accelerated IBM Power Systems servers. It includes the following frameworks:
| Framework | Version | Description |
|---|---|---|
| Caffe | 1.0 | Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by Berkeley AI Research and by community contributors. |
| Pytorch | 1.3.1.0 | Pytorch is an open-source deep learning platform that provides a seamless path from research prototyping to production deployment. It is developed by Facebook and by community contributors. |
| TensorFlow | 2.1.14.0 | TensorFlow is an end-to-end open-source platform for machine learning. It is developed by Google and by community contributors. |
...
| Code Block |
|---|
module load wmlce/1.6.1-py2.7 # for python2 environment2 module load wmlce/1.6.1-py3.6 # for python3 environment module load wmlce # python3 environment by default7.0 |
Install samples for Caffe:
...
| Code Block |
|---|
module load wmlce/1.6.1-py2.7 # for python2 environment2 module load wmlce/1.6.1-py3.6 # for python3 environment module load wmlce # python3 environment by default7.0 |
Install samples for Caffe2:
...
| Code Block |
|---|
module load wmlce/1.6.1-py2.7 # for python2 environment2 module load wmlce/1.6.1-py3.6 # for python3 environment module load wmlce # python3 environment by default7.0 |
Copy the following code into file "mnist-demo.py":
...
| Code Block |
|---|
module load wmlce/1.6.1-py2.7 # for python2 environment2 module load wmlce/1.6.1-py3.6 # for python3 environment module load wmlce # python3 environment by default7.0 |
Download the code mnist-with-summaries.py to $HOME folder:
...
After job completed the TensorFlow log files can be found in "~/tensorflow/mnist/logs", start the TensorBoard server on login node:
| Code Block |
|---|
module load wmlce
tensorboard --logdir ~/tensorflow/mnist/logs/ --port <user_pick_port> # please use random number within [6500-6999] |
Forward the <user_pick_port> on remote machine to the port <user_pick_port> on local machine:
| Code Block |
|---|
ssh -L <user_pick_port>:<node_name>:<user_pick_port> <user_name>@hal.ncsa.illinois.edu |
Paste the following address into a web browser to start the TensorBoard session:
...
hal-ondemand, detail refers Getting started with HAL OnDemand.
Simple Example with Pytorch
...
| Code Block |
|---|
module load wmlce/1.6.1-py2.7 # for python2 environment2 module load wmlce/1.6.1-py3.6 # for python3 environment module load wmlce # python3 environment by default7.0 |
Install samples for Pytorch:
...
| Name | Version | Description |
|---|---|---|
| caffe | 1.0 | Caffe is a deep learning framework made with expression, speed, and modularity in mind. |
| cudatoolkit | 10.12.16889 | The NVIDIA® CUDA® Toolkit provides a development environment for creating high-performance GPU-accelerated applications. |
| cudnn | 7.6.5.1+10.12 | The NVIDIA CUDA® Deep Neural Network library (cuDNN) is a GPU-accelerated library of primitives for deep neural networks. |
| nccl | 2.45.76 | The NVIDIA Collective Communications Library (NCCL) implements multi-GPU and multi-node collective communication primitives that are performance-optimized for NVIDIA GPUs. |
| opencv | 3.4.28 | OpenCV was designed for computational efficiency and with a strong focus on real-time applications. |
| pytorch | 1.3.1.0 | PyTorch enables fast, flexible experimentation and efficient production through a hybrid front-end, distributed training, and ecosystem of tools and libraries. |
| tensorboard | 2.1.14.0 | To make it easier to understand, debug, and optimize TensorFlow programs, we've included a suite of visualization tools called TensorBoard. |
| tensorflow-gpu | 2.1.14.0 | The core open-source library to help you develop and train ML models. |
...