Content

Code execution

To execute your code you need to

have a binary (executable, model code), which is the result of code compilation,
create a slurm job script,
submit the slurm jobs script.

Job submission

blogin> sbatch myjobscipt.slurm
Submitted batch job 8028673
blogin> ls slurm-8028673.out
slurm-8028673.out

Slurm scripts for mpirun

A slurm script is submitted to the job scheduler slurm. It contains

the control about requested compute nodes and
commands to start your binary. Using mpirun the pinning is controlled by the MPI library. Pinning by slurm you need to switch off by adding export SLURM_CPU_BIND=none.

MPI only

MPI, full node

#!/bin/bash
#SBATCH --nodes=2
#SBATCH --partition=cpu-clx:test
module load openmpi/gcc/5.0.3
export SLURM_CPU_BIND=none
mpirun -np 192 --map-by ppr:96:node ./hello.bin

MPI, half node

#!/bin/bash
#SBATCH --nodes=2
#SBATCH --partition=cpu-clx:test
module load openmpi/gcc/5.0.3
export SLURM_CPU_BIND=none
mpirun -np 96 --map-by ppr:48:node ./hello.bin

MPI, OpenMP

You can run one code compiled with MPI and OpenMP. The examples cover the setup

2 nodes,
4 processes per node, 24 threads per process.

MPI, OpenMPI, full node

#!/bin/bash
#SBATCH --nodes=2
#SBATCH --partition=cpu-clx:test
module load openmpi/gcc/5.0.3
export SLURM_CPU_BIND=none
export OMP_NUM_THREADS=24
mpirun -np 8 --map-by ppr:4:node:pe=24 ./hello.bin

Workflow OpenMPI

Code execution

Slurm scripts for mpirun

MPI only

MPI, OpenMP