Our Next-Generation Technology Pool of systems serves for the exploration and evaluation of new technologies for HPC and AI workloads.
Policies
Notes on general usage concept (proposal bySteffen Christgau (Unlicensed) ): Since no resource management system is currently installed, be gentle to other users, take care to not disturb experiments/measurements of others.
Systems
Hosts with Intel Optane Memory
Hostnames: apass{1,2}
The Apache Pass Systems are equipped with Intel Optane Memory components, both as Storage Class Memory (SCM/NVRAM) and SSD. The main difference between the two hosts is the memory capacity.
The Apache Pass memory, i.e. each of the systems, can be configured in two (three) modes:
- Memory Mode: The Apache Pass/Optane Memory is exposed as RAM to the OS. Although the hardware is different, this mode allows transparent usage of this memory type. The main benefits are the gained memory capacity and easy usage (no modification of applications required). The DRAM memory acts as cache for Optane Memory. In this mode, the SCM is effectively not persistent.
- AppDirect Mode: The Apache Pass/Optane Memory is exposed as block device(s) to the OS (usually as /dev/pmemX, depending on actual configuration). On such block devices, a file system can be created which should support the direct access (DAX) option. This enables mapping data on the persistent Optane Memory into an application's virtual address space while avoiding the OS' page cache. Thus, direct media access with load/store operations is possible. Note that data on the Optane Memory is deleted when the mode is changed. Therefore, data on /dev/pmem should be considered as ephemeral.
- (Mixed/Hybrid Mode: The System can also be configured to provide a portion of the Optane capacity for the Memory Mode and another portion for AppDirect mode.)
By default: apass1 is configured in Memory Mode while apass2 is configured in AppDirect Mode. (tbd) If you need a different configuration contact ...who....
As simple check for the given mode you can run free -h. If the total memory capacity is around or larger than 3 TB the system is in memory mode. Further, if /dev/pmem[01] exists, the AppDirect (or Mixed/Hybrid) mode is in effect.
Hardware
| CPU | 2x Intel Xeon Platinum 8260L (24c, 2,4 GHz) Cascade Lake SP |
| System | Inspur NF5280M5 |
| Memory | apass1:
apass2:
All DIMM slots fully populated with Optane/DRAM pairs (2:2:2 configuration). The Optane DIMMs are interleaved and a single region spans over them (per socket) |
| Storage | apass1:
apass2:
|
| Network | Single Port Omni-Path HFI Adapter 100 Series (back-to-back connected via Cu cable) |
Software
- OS: Centos7
- more recent software (compilers, libraries, utilities) are available via environment modules (module avail). Module files are stored in
/opt/local/modules.
NEC SX-Aurora TSUBASA A300-8
Hostname: aurora
NOTE: System software currently updated.
Hardware Configuration
| Host CPU | 2x Intel Xeon Gold 6126 (12c, 2,6 GHz) Skylake |
| Host Memory | 192 GB (DDR4-2666 ESS RDIMM) |
| Accelerators | 8x NEC Vector Engines 1.0 (VE) Modell B |
| VE Configuration | per VE:
|
| Network | 2x 100 Gb/s IB between the two PCI root complexes |
Software
| Host OS | CentOS 7.6 |
| VE OS | tb |
| NEC Compiler Suite | tb |
Access
To gain access to the ...PLATFORMNAME..., contact support@hlrn.de. Please give a short description of your intention and the system you want to use.