Starting from quarter 04/2022, the two NHR centers
- NHR@ZIB and
- NHR@Göttingen
follow the NHR-wide regulations and account in the new unit core hours.
Both NHR centers provide computational resources primarily to universities and scientific institutions in Germany under public law.
to be updated: The details regarding the policies for resource allocation and payment are prescribed by the Fees And Regulations on the HLRN webpage Zulassungs- und Entgeltordnung considering the unit conversion 1 NPL = 6.86 core hour.
- A user accesses an account containing a number of units core hour.
- The NHR centers charge for batch jobs on the compute systems. The charge for a job is a number of units core hour.
- Usage of persistent storage magnetic tapes or hard disk capacity is currently not accounted.
Charge rates
The NHR centers operate different types of compute nodes (HLRN-IV system) organized in partitions, see Hardware. The charge rates for the partitions are given in the table.
...
standard96
standard96:test
...
large96
large96:test
large96:shared
...
192*
...
medium40
medium40:test
...
large40
large40:test
...
*increased charge rate due to high memory layout
Job charge
The charge for a batch job on a HLRN system is a number of NPLs and is calculated from the number of nodes reserved for the jobContent
| Inhalt |
|---|
Project account
NHR center NHR@ZIB follows NHR-wide regulations.
- A User Account accesses a project account containing units of core hour. A project account can be a Test Project or a Compute Project.
- A batch job on the compute system is charged by a number of core hour to measure the usage.
- Usage of persistent storage including the tape library are currently not accounted.
Charge rates
NHR@ZIB operates system Lise with different Compute partitions. Properties for available (slurm-)partitions you find on the pages
Each partition holds a specific charge rate.
| Compute partition | Slurm partition | Charge (core hour) per 1 node per 1 h occupancy time | Remark |
|---|---|---|---|
cpu-clx | 96 | ||
| cpu-clx:test | |||
| cpu-clx:ssd | |||
cpu-clx:large | 144 | high memory layout | |
| cpu-clx:huge | 192 | high memory layout | |
| CPU Genoa partition | cpu-genoa cpu-genoa:test | free of charge1 | test phase |
| GPU A100 partition | gpu-a100 | 600 | four NVidia A100 (80 GB) per compute node |
gpu-a100:shared | 150 per GPU | 600 for four NVidia A100 (80 GB) per node | |
gpu-a100:shared:mig | 21.43 per MiG slice | four NVidia A100 (80 GB) splitted each into two 2g.10gb slices (8 per node and currently 24 in total) and one 3g.20gb slice (4 per node and currently 12 in total) | |
| GPU PVC partition | gpu-pvc | free of charge1 | test phase |
1: Practically the charge is a very small number (close to zero).
Job charge
The charge of core hours for a batch job depends on the number of nodes, the wallclock time used by the job, and the charge rate for the job nodespartition used. For a batch job with
num a number of nodes n,
running with a wallclock time of t hours, and
- on a partition with a charge rate charge_p
the job charge charge_j yields
charge_j = numn * t * charge_p
| Panel | ||
|---|---|---|
| ||
A job on 10 nodes running for 3 hours on partition huge96 (= 192 core hour) yields a job charge of 5760 core hour. |
...
| Panel | ||
|---|---|---|
| ||
A job on 48 cores on partition large96:shared (96 cores per node, 144 core hour) has a reservation for num = 48/96 = 0.5 nodes. Assuming a wallclock time of 3 hours yields a job charge of 216 core hour. |
Accounts for core hour
Running batch jobs with a user account, the user can access different accounts in general. The NHR centers distinguish between
- personal accounts and
- project accounts
User account
In the HLRN each user account is attached to a personal account. Both accounts, the user account and the personal account, share the same name but their purpose is different. The user account is the account on the Linux operation system and is used to control e.g. permissions to data files. The personal account is a bank account containing core hour.
...
core |
...
hour |
...
. |
...
Project account
A compute project holds a project account. The project account is a bank account containing core hour for the project. At the beginning of each quarter the account is granted by the number of core hour following the funding decision for the compute project. Unused core hour are transferred to the subsequent quarter, but only one time.
In case of problems with core hour in the project account please look at /wiki/spaces/PUB/pages/425995 or contact your consultant. This might affect the
- application for additional core hour,
- movement of core hour between quarters.
Batch job and account
Batch jobs are submitted by a user account to the compute system. For each job the user chooses the account that will be charged by the job.
For the user account the default account for compute time is defined on the link Informationen zu Ihrer Kennung/Account information of the service portal. At the beginning of the user account lifetime the default account is the personal account.
The user controls the account for a job using the Slurm option --account at submit time.
| Codeblock | ||
|---|---|---|
| ||
To charge the account myaccount
add the following line to the job script.
#SBATCH --account=myaccount
|
After job script submission the batch system checks the account for account coverage and authorizes the job for scheduling. Otherwise the job rejected.
| Codeblock | ||
|---|---|---|
| ||
| You can check the account of a job that is out of core hour. > squeue ... myaccount ... AccountOutOfNPL ... |