BinAC/Quickstart Guide

1 File System Basics

The details of the file systems are explained here.

1.1 Home File System

Home directories are meant for permanent file storage of files that are keep being used like source codes, configuration files, executable programs, conda environments, etc. It is backuped daily and has a quota.

1.2 Work File System

Use the work file system and not your home directory for your calculations. Create a working directory using your username.

cd /beegfs/work/
mkdir <username>
cd <username>

Do not use the login nodes to carry out any calculations or heavy load file transfers.

1.3 Temporary Data

If your job creates temporary data, you can use the fast SSD with a capacity of 211 GB on the compute nodes. The temporary directory for your job is available with the $TMPDIR environment variable.

2 Queue Basics

The most recent Queue settings are displayed on login as message of the day on the terminal.

Get an overview of the number of running and queue jobs:

$ qstat -q

Queue            Memory CPU Time Walltime Node  Run Que Lm  State
---------------- ------ -------- -------- ----  --- --- --  -----
tiny               --      --       --      --    0   0 --   E R
long               --      --       --      --  850   0 --   E R
gpu                --      --       --      --   66   0 --   E R
smp                --      --       --      --    4   1 --   E R
short              --      --       --      --  131  90 --   E R
                                               ----- -----
                                                1051    91

To check all running and queued jobs.

qstat

Just your own jobs.

qstat -u <username>

3 Simple Interactive Job

To start a 1 core job on a compute node providing a remote shell.

qsub -q short -l nodes=1:ppn=1 -I

The same but requesting the whole node.

qsub -q short -l nodes=1:ppn=28 -I

Standard Unix commands are directly available, for everything else use the modules.

module avail

Just an example

module load chem/gromacs/4.6.7-gnu-4.9
g_luck

Be aware that we allow node sharing. Do not disturb the calculations of other users.

4 Simple Script Job

Use your favourite text editor to create a script.

#PBS -l nodes=1:ppn=1
#PBS -l walltime=00:05:00
#PBS -S /bin/bash
#PBS -N Simple_Script_Job
#PBS -j oe
#PBS -o LOG
cd $PBS_O_WORKDIR
echo "my Username is:"
whoami
echo "My job is running on node:"
uname -a
module load chem/gromacs/4.6.7-gnu-4.9
g_luck

Submit the job using

qsub -q short script.sh

Take a note of your jobID.

5 Killing a Job

Let's assume you build a Homer and want to stop/kill/remove a running job.

qdel <jobID>

6 Fancy Script Job

#PBS -l nodes=1:ppn=28:gpus=4:exclusive_process
#PBS -l walltime=36:00:00
#PBS -S /bin/bash
#PBS -N Gromacs_GPU
#PBS -j oe
#PBS -o LOG
#PBS -n
module purge
module load chem/gromacs/2016.4-gnu-5.2
cd $PBS_O_WORKDIR
gmx grompp -f NPT.mdp -c protein.pdb -n index.ndx -p topol.top
mdrun_s_gpu -v -deffnm NPT_protein -pin on -ntmpi 4 -ntomp 7 -gpu_id 0123 -s topol.tpr

Submit with

qsub -q gpu script.sh

There are tons of options, details and caveats. If there is anything not working, as you like, send an email to hpcmaster@uni-tuebingen.de.