Versions Compared

Old Version 10

changes.mady.by.user Volodymyr Kindratenko

Saved on

compared with

New Version Current

changes.mady.by.user Volodymyr Kindratenko

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In publications and presentations that use results obtained on this system, please include the following acknowledgement: “This work utilizes resources supported by the National Science Foundation’s Major Research Instrumentation program, grant #1725729, as well as the University of Illinois at Urbana-Champaign”.

System Description

Host name: hal-dgx.ncsa.illinois.edu

Hardware

Software

  • CentOS 7RHEL 8
  • CUDA 11
  • NCCL
  • cuDNN
  • TensorRT12



User Guide

To request an account: email kindrtnk@illinois.edu

Note

Use /home/<username> for basic stuff only, do not put any code/data here as the /home partition is very small. Quota: 50GB per User

Use /raid partition projects file system for all your data/code. Create a subfolder in this partition for your project username and keep your stuff there. Quota: 2TB/10 million inodes per User

Use /scratch file system for ephemeral/transient data that is created during job runs that doesn't need to be kept long term.

To compile CUDA applications, first enable devtoolset 9

Code Blockscl enable devtoolset-9 bash

(And do not forget to remove it after the job is done and you no longer need it!)

To access the system, submit jobs, etc.: Access HAL-DGX and OVERDRIVE servers through HAL-LOGIN4 node

CUDA tools are already in the path, just use nvcc to compile.

To work with Python

, create

, there are several versions installed, from 2.7 to 3.9. Create and enable virtual environment first, e.g., to use 3.8:

Code Block
python3.8 -m venv python3python38-virtualenv
source python3python38-virtualenv/bin/activate

You can use pip to install python packages within this environment.

To start a Jupyter notebook on hal-dgx

  • on hal-dgx:
Code Block
# source python3 your python environment, e.g., 3.8
source python38-virtualenv/bin/activate
pip install jupyter # this needs to be done only once
jupyter notebook --port=9999 # this will start the jupyter server on port 9999. Pick a different port if it fails to start
  • on your own computer:
Code Block
# This opens a connection to the hal-dgx.ncsa.illinois.edu Jupyter server, and 
# forwards any connection to port 8888 on the local machine to port 88889999 on hal-dgx.ncsa.illinois.edu.
ssh -L 8888:localhost:88889999 hal<userid>@hal-dgx.ncsa.illinois.edu
  • Finally, on your own computer, open web browser and point it to the address you see after running 'jupyter notebook' on hal-dgx, something like like http://localhost:8888/?token=...
    • Make sure to replace the port in the URL with the port on your localhost

Main -> Systems -> DGX A100

Contact us

Request access to ISL resources: Application

Contact ISL staff: Email Address

Visit: NCSA, room 3050E