Architecture Diagram

SSU (Scaleable Storage Unit)

The building block for the Taiga system is known as an SSU (Scaleable Storage Unit).  Each SSU is composed of a DDN ES400NVXE unit containing 12 x 15.36TB NVME drives, with 4 SS9012 enclosures attached containing a total of 352 18TB SAS drives.  This gives each SSU a capacity of ~5PB usable after formatting and data protection.  Two SSUs are housed per rack, with room at the top for service nodes, DTN nodes, and switches.  Taiga is built to scale out with more of these SSUs as demand for the system grows over time.  

Interconnect Fabric

Taiga is built on a Mellanox HDR fabric with an aggregate bandwidth of 4TB/s.  All DDN controller couplets, cluster LNET routers, and Cluster Export Services nodes connect to a pair of Mellanox QM8700 40-port Managed HDR switches.  This fabric serves as the backbone for the storage infrastructure.  LNET routers placed in each system provide the network translation to their own fabric type.  

File System

DDN's ExaScaler Lustre appliance was chosen for use on Taiga currently running Exa 6 based on Lustre 2.14.  Lustre is a highly scaleable and performant HPC file system with a growing number of features that help support bare metal workloads but also VMs and Containers.  Newer Lustre features such as sub-directory mount, PFL, and DNE3 are being leveraged on Taiga in addition to tools and features provided by DDN's ExaScaler platform such as snapshots and the Stratagem policy engine.  

  • No labels