Archive Architecture

The archive is essentially made up of dour main parts, the Granite servers, the disk cache, and the Library, and it's export nodes. The Granite servers connect to the library via 2Gb FC, the cache via 12G SAS, and the NFS Export nodes via 2x100 GbE.

Block Size on Cache

File System Block Size: 1 MB

For a balance of throughput performance and file space efficiency, a block size of 1MB has been chosen for the ScoutFS file system that acts as the Granite front end disk cache.

Section Size on Tape

The ScoutAM software allows for section sizing media into chunks to mitigate writing small files to tape. We currently have a section size of 10GB.

Accessing Granite:

There are currently three different methods that will be implemented to store and retrieve data, Globus, SCP, and a DR Solution the SET team provides.

Globus:

Globus is a third party service NCSA uses to leverages across a number of it's storage systems.

Granite is accessible via Globus at the endpoint name "NCSA Granite" and the endpoint is open to the public internet for transfers from anywhere with another Globus endpoint.  Authentication to the endpoint is handled by NCSA's CILogon service and requires two-factor via Duo.  For shared collections, or other questions submit a ticket to help+globus@ncsa.illinois.edu.  Information about Globus can be found at their site https://www.globus.org

SCP:

This method leverages the native linux secure copy tool scp to ingest data into the archive cache. 

DR-Backup

Usage and Quota controls

Capacity to inode Ratio

Ratio: 1TB Quota to 625,000 iNodes

An inode is a record that describes a file, directory, or link.  This information is stored in a dedicated flash pool on Taiga and is of a finite capacity.  To ensure that the inode pool does not run out of space before the capacity pools, this quota ratio is implemented.  So for example if your project has a 10TB quota on Taiga, it will also have a quota of 6.25 million inodes (I.E. You can have 6.25 million different files and directories). If you have any questions about this ratio please contact the storage team by opening a support ticket.  

Controlling Quota

Quota is currently controlled physically. ScoutAM and the filesystem it's based off (ScoutFS) currently only allows to assign a pool of tapes to users/groups. Each tape is 4TB so quota's will be set in 4TB chunks up to and slightly over (1-3GB) of your allocated quota. We intend for this to be controlled more precisely at the filesystem level in the near future.