Versions Compared

Old Version 49

changes.mady.by.user Yan Zhan

Saved on

compared with

New Version 148

changes.mady.by.user Yan Zhan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

"My name is HAL. I became operational on March 25 2019 at the Innovative Systems Lab in Urbana, Illinois. My creators are putting me to the fullest possible use, which is all I think that any conscious entity can ever hope to do." (

paraphrazed from

paraphrased from https://en.wikipedia.org/wiki/HAL_9000)

In publications and presentations that use results obtained on this system, please include the following acknowledgement: “This work utilizes resources supported by the National Science Foundation’s Major Research Instrumentation program, grant #1725729, as well as the University of Illinois at Urbana-Champaign”.

Also, please include the following reference in your publications: “V. Kindratenko, D. Mu, Y. Zhan, J. Maloney, S. Hashemi, B. Rabe, K. Xu, R. Campbell, J. Peng, and W. Gropp. HAL: Computer System for Scalable Deep Learning. In Practice and Experience in Advanced Research Computing (PEARC ’20), July 26–30, 2020, Portland, OR, USA. ACM, New York, NY, USA, 15 pages. https://doi.org/10.1145/3311790.3396649”.

Hardware-Accelerated Learning (HAL) cluster


Info

Effective May 19, 2020, two-factor authentication via NCSA Duo is now required for SSH logins on HAL. See https://go.ncsa.illinois.edu/2fa for instructions to sign up.

System Description


Host name: hal.ncsa.illinois.edu

Hardware

GB
ZFS array, NFS-mounted on all nodes via IB EDR

Software

Documentation

Science on HAL

Software for HAL

  • Storage upgrade TBD
    • Software

    Documentation

  • Job management with SLURM
  • Modules management
  • Getting started with PowerAI
  • Using Jupyter Notebook on HAL
    • Working with containers


    To request access: fill out this form. Make sure to follow the link

    on

    in the

    application

    confirmation

    page

    email to request

    an

    actual system account.

    Frequently Asked Questions

    To report problems: email us

    For our new users: New User Guide for HAL System

    User group Slack space: https://join.slack.com/t/halillinoisncsa

    Real-time Dashboards: Here

    HAL OnDemand portal: https://hal-ondemand.ncsa.illinois.edu/

    Globus Endpoint: ncsa#hal

    Quick start guide: (for complete details see Documentation section on the left)

    To connect to the cluster:

    Code Block
    ssh
    username@hal
    <username>@hal.ncsa.illinois.edu

    To submit interactive job:

    Code Block
    languagebash
    srun
    swrun -
    -partition=debug --pty --nodes=1 \ --ntasks-per-node=8 --gres=gpu:v100:1 \ -t 01:30:00 --wait=0 \ --export=ALL /bin/bash 
    p gpux1

    To submit a batch job:

    Code Block
    sbatch
    swbatch run_script.
    sb
    swb
    See run_script.sb for a basic example.
    Job Queue time limits:
    Interactive 
    • "debug" queue: 
     12 
    • 4 hours
  • Batch queue:  72 hours
    • To load IBM PowerAI module
    • "gpux<n>" and "cpun<n>" queues:  24 hours
    Resource limits:
    • 5 concurrently running jobs
    • concurrently allocated resources
      • 5 nodes
      • 16 GPUs
    • For larger/more numerous jobs, please contact admins for a special arrangement and/or a reservation
    To load the OpenCE module (provides PyTorch, Tensorflow and other ML tools):
     Code Block
    module load 
    ibm/powerai
    opence
    To see CLI scheduler status:
     Code Block
    swqueue


    Main -> Systems -> HAL
    Contact us
    Request access to ISL resourcesthis system: Application
    Contact ISL staff: Email Address
    Visit: NCSA, room 3050E

    Image RemovedImage Added