Preface
CentOS 7 has reached its end of life. For this reason the operating system (OS) of Lise's CPU partition will be updated to Rocky Linux 9. This affects all login and compute nodes equipped with Intel Xeon Cascade Lake processors ("clx" for short). Lise's GPU partitions ("a100" and "pvc") are not affected.
Rocky Linux 9 introduces new versions of various system tools and libraries. Some codes compiled earlier under CentOS 7 might not be working under Rocky Linux 9 anymore. Thus, legacy versions of environment modules offered under CentOS 7 were not transferred to the new OS environment or have been replaced by more recent versions. To adapt to the new OS environment quickly, it is important for users to read this page and to follow the action items specified below.
The OS migration phase starts with two login nodes and one rack of 96 compute nodes already running Rocky Linux 9 for testing, while the other nodes remain available under CentOS 7. After an initial testing period, a major fraction of nodes will be switched to Rocky Linux 9 to allow for general job production under the new OS. In the last step, the few nodes still under CentOS 7 will be converted to Rocky Linux 9, too. During the migration phase the use of Rocky Linux 9 "clx" compute nodes will be free of charge.
Current migration state
nodes | CentOS 7 | Rocky Linux 9 |
---|---|---|
login | blogin[1-6] | blogin[7-8] |
compute (384 GB RAM) | 852 | 96 |
compute (768 GB RAM) | 32 | 0 |
compute (1536 GB RAM) | 2 | 0 |
Latest news
date | subject |
---|---|
2024-07-02 | announcement: start of migration phase with 2 login and 96 compute nodes running Rocky Linux 9 |
What has changed
Software and environment modules
CentOS 7 | Rocky Linux 9 | |
---|---|---|
OS components | glibc 2.17 | glibc 2.34 |
Python 3.6 | Python 3.9 | |
GCC 4.8 | GCC 11.4 | |
bash 4.2 | bash 5.1 | |
Environment modules version | 4.8 (Tmod) | 5.4 (Tmod) |
Modules loaded initially |
|
|
|
| |
|
| |
compiler modules |
|
|
|
| |
MPI modules |
|
|
|
|
Shell environment variables
CentOS 7 | Rocky Linux 9 |
---|---|
| (undefined, local |
|
|
(undefined) |
|
(undefined, defaults to |
|
(undefined) |
|
(undefined) |
|
(undefined) |
|
SLURM partitions
CentOS 7 | Rocky Linux 9 |
---|---|
|
|
|
|
|
|
|
|
| |
| |
|
|
What remains unchanged
node hardware and node names
communication network (Intel Omnipath)
file systems (HOME, WORK, PERM) and disk quotas
environment modules system (still based on Tcl, a.k.a. “Tmod”)
access credentials (user IDs, SSH keys) and project IDs
charge rates and CPU time accounting (early migrators' jobs are free of charge)
Lise’s Nvidia-A100 and Intel-PVC partitions
Special remarks
For users of SLURM’s
srun
job launcher:
Open MPI 5.x has dropped support for the PMI-2 API, it solely depends on PMIx to bootstrap MPI processes. For this reason the environment setting was changed fromSLURM_MPI_TYPE=pmi2
toSLURM_MPI_TYPE=pmix
, so binaries linked against Open MPI can be started as usual “out of the box” usingsrun mybinary
. For the case of a binary linked against Intel-MPI, this works too when a recent version (≥2021.11) of Intel-MPI has been used. If an older version of Intel-MPI has been used, and relinking/recompiling is not possible, one can follow the workaround for PMI-2 withsrun
as described in the Q&A section below. Switching fromsrun
tompirun
instead should also be considered.
Action items for users
All users of Lise are recommended to
log in to an already migrated login node and get familiar with the new environment
check self-compiled software for continued operability
relink/recompile software as needed
adapt and test job scripts and workflows
submit test jobs to the "cpu-clx:test" SLURM partition
read the Q&A section and ask for support in case of further questions, problems, or software requests (support@nhr.zib.de)