JUSTUS2/Software/Singularity

Description	Content
License	Open-source software, distributed under the 3-clause BSD License. More...
Citing	---
Links	Homepage | Documentation
Graphical Interface	No

Description

Singularity is a container platform.

License

Singularity is free, open-source software released under the 3-clause BSD license. Please read the license for additional information about Singularity.

Usage

Availability

Singularity is directly available on all compute nodes (but not on the login nodes). You do not have to load a module, but you must use a compute node i.e. via an interactive job

The binary singularity is main program of the container platform.

To get help using Singularity execute the following command:

$ singularity --help

Furthermore, a man page is available and can be accessed by typing:

$ man singularity

For additional information about how to use Singularity, please consult the documentation.

Batch jobs with containers

Batch jobs utilizing Singularity containers are generally built the same way as all other batch jobs, where the job script contains singularity commands. For example:

#!/bin/bash
# Allocate one node
#SBATCH --nodes=1
# Number of program instances to be executed
#SBATCH --tasks-per-node=4
# 8 GB memory required per node
#SBATCH --mem=16G
# Maximum run time of job
#SBATCH --time=1:00:00
# Give job a reasonable name
#SBATCH --job-name=Singularity
# File name for standard output (%j will be replaced by job id)
#SBATCH --output=singularity_job-%j.out
# File name for error output
#SBATCH --error=singularity_job-%j.err

cd your/workspace

# Run container (two options to start a container)
singularity run [options] <container>
singularity exec [options] <container> <command>

Keep in mind that other modules you may have loaded will not be available inside the container.

Using GPUs

#!/bin/bash
[…]
# Allocate one GPU per node
#SBATCH --partition=gpu
#SBATCH --gres=gpu:1
[…]

cd your/workspace

# Run container (two options to start a container)
singularity run --nv [options] <container>
singularity exec --nv [options] <container> <command>

Using the flag is advisable, but may be omitted if the correct GPU- and driver-APIs are available on the container.

Examples

Run your first container on JUSTUS 2

Build a TensorFlow container with Singularity and execute a Python command:

# request interactive node with GPUs
$ srun --nodes=1 --exclusive --gres=gpu:2 --pty bash

# create workspace and navigate into it
$ WORKSPACE=`ws_allocate tensorflow 3`
$ cd $WORKSPACE

# build container
$ singularity build tensorflow-20.11-tf2-py3.sif docker://nvcr.io/nvidia/tensorflow:20.11-tf2-py3

# execute Python command
$ singularity exec --nv tensorflow-20.11-tf2-py3.sif python -c 'import tensorflow as tf; \
  print("Num GPUs Available: ",len(tf.config.experimental.list_physical_devices("GPU")))'

Note: Ready-to-use containers can be pulled from the NVIDIA GPU CLOUD (NGC) catalog.

Batch jobs with containers on JUSTUS 2

Run a GROMACS container with Singularity as a batch job:

$ salloc --ntasks=1                                                         # obtain compute node
$ WORKSPACE=`ws_allocate gromacs 3`                                         # allocate workspace
$ cd $WORKSPACE                                                             # change to workspace
$ singularity pull gromacs-2020_2.sif docker://nvcr.io/hpc/gromacs:2020.2   # pull container from NGC
$ cp -r /opt/bwhpc/common/chem/ngc/gromacs/ ./bwhpc-examples/               # copy example to workspace
$ cd ./bwhpc-examples                                                       # change to example directory
$ sbatch gromacs-2020.2_gpu.slurm                                           # submit job
$ squeue                                                                    # obtain JOBID
$ scontrol show job <JOBID>                                                 # check state of job

More batch job examples are located at /opt/bwhpc/common/chem/ngc.

Useful links

• Documentation (english)
• Wikipedia article (german)
• Wikipedia article (english)

---