File Systems at HLRN-IV

...

HLRN is operating at each site 3 central storage systems:

• IBM Spectrum Scale file system with 340 TiByte capacity containing , which exports via NFS
  • $HOME directories /home/${USER}/
  • Centrally managed software and the module system in /sw
• Lustre parallel file system with 8 .1 PiByte capacity containing
  • $WORK directories /scratch/usr/${USER}/
  • $TMPDIR directories /scratch/tmp/${USER}/
  • project data directories /scratch/projects/<projectID>/ (not yet available)
• Tape archive with 120 TiByte HDD cache (accessible on the login nodes, only (Lise), access using gperm1/2 on Emmy)On Emmy: (multiple Petabyte) with additional harddisk caches

The system Emmy has addtional storage options for high IO demands:

• Phase 1 nodes (partitions medium40 and large40): Local SSD for temporary data at $LOCAL_TMPDIR (400 GB shared among all jobs running on the node). The environment variable $LOCAL_TMPDIR is available on all nodes, but on the phase 2 systems it points to a ramdisk.
• DDN IME based burst buffer with 48TiB NVMe storage (general availability together with the phase 2 nodes)

...

For reasons of efficiency and performance, many small files and/or complex directory structures should not be transferred here directly. Compressed tarballs containing such data are preferred.

access using gperm1/2 on Emmy)