JUSTUS2/Software/Turbomole

Description	Content
module load	chem/turbomole
Availability	bwUniCluster | BwForCluster_Chemistry
License	Commerical
Citing	See Turbomole manual
Links	Homepage | Documentation
Graphical Interface	No (Yes, for generating input)
User Forum	external

1 Description

Turbomole is a general purpose quantum chemistry software package for ab initio electronic structure calculations and provides:

• ground state calculations for methods such as Hartree-Fock, DFT, MP2, and CCSD(T);
• excited state calculations at different levels such as full RPA, TDDFT, CIS(D), CC2, an ADC(2);
• geometry optimizations, transition state searches, molecular dynamics calculations;
• property and spectra calculations such as IR, UV/VIS, Raman, and CD;
• approximations like resolution-of-the-identity (RI) to speed-up the calculations without introducing uncontrollable or unknown errors; as well as
• parallel versions (OpenMP, Fork, MPI and Global Arrays) for almost all kind of jobs.

For more information on Turbmole's features please visit http://www.turbomole-gmbh.com/program-overview.html.

2 Versions and Availability

A current list of the versions available on the bwUniCluster and bwForClusters can be obtained from the
Cluster Information System: CIS Information on Turbomole or here: {{#widget:Iframe |url=https://cis-hpc.uni-konstanz.de/prod.cis/bwUniCluster/chem/turbomole |width=99% |height=330 |border=0 }} On the command line interface (CLI) of a particular bwHPC cluster a list of all available Turbomole versions can be inquired as followed

$ module avail chem/turbomole

2.1 Parallel computing

The Turbomole Module subsumes all available parallel computing variants of Turbomole's binaries. Turbomole defines the following parallel computing variants:

• SMP = Shared-memory parallel computing based on OpenMP and Fork() with the latter using separated address spaces.
• MPI = Message passing interface protocol based parallel computing
• GA = Global arrays, API for "shared-memory" programming for distributed-memory computers which can be used e.g. to complement MPI.

However only one of the 3 parallel variants or the sequential variant can be loaded at once and most Turbomole's binaries support only 1 or 2 of the parallelization variants. Like for Turbomole installations without a Module system, the variants have to be triggered by the environment variable $PARA_ARCH.

3 Usage

3.1 Before loading the Module

Before loading the Turbomole Module the parallel computing variant has to be defined via the environment variable $PARA_ARCH using the abbreviations SMP, MPI or GA, e.g.:

$ export PARA_ARCH=MPI

will later load the MPI binary variants. If the variable $PARA_ARCH is not defined or empty, the sequential binary variants will be active once the Turbomole Module is loaded.

3.2 Loading the Module

You can load the default version of Turbomole with the command:

$ module load chem/turbomole

The Turbomole Module does not depend on any other Module, but on the variable $PARA_ARCH. Moreover, Turbomole provides its own libraries regarding OpenMP, Fork(), MPI, and Global Array based parallelization. If you wish to load a specific (older) version you can do so by executing e.g.:

$ module load chem/turbomole/6.5

to load the version 6.5

3.3 Switching between different parallel variants

To switch between the different parallel variants provided by the Turbomole Module, simply define the new parallel variant via $PARA_ARCH and load the Module again. Note that for switching between the parallel variants unloading of the Turbomole Module is not required. For instance to change to the MPI variant, execute:

$ export PARA_ARCH=MPI
$ module load chem/turbomole

3.4 Turbomole binaries

The Turbomole software package consists of a set of stand-alone program binaries providing different features and parallelization support:

Binary	Features	OpenMP	Fork	MPI	GA
define	Interactive input generator	no	no	no	no
dscf	Energy calculations	yes	yes	yes	no
grad	Gradient calculations	no	yes	yes	no
ridft	Energy calc. with fast Coulomb approximation	no	yes	yes	yes
rdgrad	Gradient calc. with fast Coulomb approximation	no	yes	yes	yes
ricc2	Electronic excitation energies, transition moments and properties of excited states	yes	no	yes	no
statpt	Hessian and coordinate update for stationary point search	no	no	no	no
aoforce	Analytic calculation of force constants, vibrational frequencies and IR intensities	no	yes	no	no
escf	Calc. of time dependent and dielectric properties	no	yes	no	no
egrad	gradients and first-order properties of excited states	no	yes	no	no
odft	Orbital-dependent energy calc.	yes	no	no	st no

For the complete set of binaries and more detailed description of their features read here.

3.5 Turbomole tools

Turbomole's tool set contains scripts and binaries that help to prepare, execute workflows (such as geometry optimisation) and postprocess results.

Script	Type	Description
x2t	Preparation	Converts XYZ coordinates into Turbomole coordinates.
sdg	Preparation	Shows data group from control file: for example$ sdg coordshows current Turbomole coordinates used.
jobex	Optimization workflow	Shell script that controls and executes automatic optimizations of molecular geometry parameters.
tm2molden	Postprocessing	Creates a molden format input file for the Molden program.
eiger	Postprocessing	Script front-end of program eigerf to display HOMO-LUMO gap and MO eigenvalues.

For the complete set of tools and more detailed description of their features read here.

3.6 Disk Usage

By default, scratch files of Turbomole binaries are placed in the directory of Turbmole binary execution. Please do not run your Turbomole calculations in your $HOME or $WORK directory.

4 Examples

4.1 Single node jobs

4.1.1 Template provided by Turbomole Module

The Turbomole Module provides a simple Moab example of Cubane (C8H8) that runs an energy relaxation via MPI parallel dscf using 4 cores on a single node and its local file system. To simply run the example do the following steps:

$ module load chem/turbomole
$ mkdir -vp ~/Turbomole-example/
$ cd ~/Turbomole-examples/
$ cp -r $TURBOMOLE_EXA_DIR/* ~/Turbmole-example/
$ msub bwHPC_turbomole_single-node_example.sh

The last step submits the job example script bwHPC_turbomole_single-node_example.sh to the queueing system. Once started on a compute node, all calculations will be done under an unique directory on the local file system of that particular compute node.

4.1.2 Geometry optimization

To do a geometry optimization of the previous job example modify bwHPC_turbomole_single-node_example.sh by replacing the following line

time dscf > dscf.out 2>&1

with

time jobex -dscf -keep 2>&1

and submit the modified script to the queueing system via msub. The Turbomole command jobex controls the call of all the required Turbomole binaries for the geometry optimization.

5 Turbomole-Specific Environments

To see a list of all Turbomole environments set by the 'module load'-command use 'env | grep TURBO'.
Or use the command 'module display chem/turbomole/(version)'.
Example (e.g. Default version)

$ module display chem/turbomole    # output is revised
-------------------------------------------------------------------
/opt/bwhpc/common/modulefiles/chem/turbomole/6.6:
[...]
TURBOMOLE_VERSION = 6.6_tmolex40 
TURBOMOLE_HOME = /opt/bwhpc/common/chem/turbomole/6.6_tmolex40 
TURBOMOLE_DOC_DIR = /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/documentation 
TURBOMOLE_EXA_DIR = /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/bwhpc-examples 
TURBOMOLE_BPR_URL = http://www.bwhpc-c5.de/wiki/index.php/Turbomole 
TEST_MODULE = /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/install/test-module.sh 
TURBOMOLE_MODE = compute 
TURBODIR = /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/TURBOMOLE_COMPUTE 
MOLE_CONTROL = /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/TURBOMOLE_COMPUTE/MoleControl_2.5 
TURBOMOLE_SYSNAME = em64t-unknown-linux-gnu 
TURBOTMPDIR = /scratch/kn_pop'your-id' 

# PATH=$PATH 
+ /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/TURBOMOLE_COMPUTE/MoleControl_2.5 
+ /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/TURBOMOLE_COMPUTE/calculate_2.4_linux64 
+ /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/TURBOMOLE_COMPUTE/thermocalc 
+ /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/COSMObuild 
+ /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/COSMOview 
+ /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/TURBOMOLE_COMPUTE/scripts 
+ /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/TURBOMOLE_COMPUTE/bin/em64t-unknown-linux-gnu 

[...]

6 Version-Specific Information

For specific information about version (version), see the information available via the module system with the command 'module help chem/turbomole/(version)'.

$ module help chem/turbomole
----------- Module Specific Help for 'chem/turbomole/6.6' ---------
This module provides the quantum chemistry program Turbomole
version 6.6 and the corresponding visualization program
TmoleX version 4.0 (see also http://www.cosmologic.de/).

Primary Turbomole and TmoleX documentation:
--------------------------------------------------------------------------------
* http://www.cosmologic.de/index.php?cosId=1542&crId=8
* http://www.bwhpc-c5.de/wiki/index.php/Turbomole
* /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/documentation/Tutorial_6-6.pdf
  /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/documentation/Tutorial-tmolex-4-0.pdf
  /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/documentation/Documentation.pdf
* Please read the pdf-documentation for details on how to do
  scientific research with Turbomole or on how to use TmoleX.

Secondary documentation of third party tools:
--------------------------------------------------------------------------------
* /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/TURBOMOLE_COMPUTE/calculate_2.4_linux64/calculate_manual.pdf
  /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/TURBOMOLE_COMPUTE/MoleControl_2.5/doc/manual.pdf
  /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/TURBOMOLE_COMPUTE/thermocalc/thermocalc_manual.pdf

Free TmoleX client download:
--------------------------------------------------------------------------------
* http://www.cosmologic.de/index.php?cosId=3016&crId=3

Preparing a calculation and using the queueing system:
--------------------------------------------------------------------------------
* Each turbomole job must run in it's own directory (a new dir for each job).
* Usually the input files 'basis', 'control', 'coord' and 'mos' must exist in
  the current directory before you can run the turbomole commands like 'dscf'.
* The input files can be created either via:
  a) command line tool 'define' (as described in 'Tutorial_6-6.pdf') or 
  b) graphical user interface 'TmoleX' (as described in 
     'Tutorial-tmolex-4-0.pdf'). 
     Please only construct and save jobs ('Save' button) via 'TmoleX', 
     but do not use the 'Run'-job buttons (since they do not
     work properly). Furthermore you can extract the Turbomole commands 
     (e.g. 'dscf', 'jobex', ...) of your job from file 'start-job' saved 
     by TmoleX and paste these commands into your queueing system script.
* Thereafter you can run the quantum chemistry calculations interactivley,
  e.g. by calling the hartree fock SCF command 'dscf' in your interactiv login
  shell. You can monitor the memory consumption or debug your input. Example:
     cp -v /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/bwhpc-examples/{coord,*basis,control,mos} ./
     time dscf > dscf.out
* Finally you can submit a series of these Turbomole commands in a batch script
  to the queueing system. An example for such a script can be found here:
     /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/bwhpc-examples/bwHPC_turbomole_single-node_example.sh
  The example includes many comments and detailed instructions
  on how to submit Turbomole jobs to the queueing system.
  Please carefully work through the example script.
* For more information on job submit script please visit:
  http://www.bwhpc-c5.de/wiki/index.php/Batch_Jobs

Using third party tools and turbomole variants:
-------------------------------------------------------------------------------
* The commands 'calculate', 'mole' (package MoleControl) and 'thermocalc'
  are available on command line after loading the turbomole module.
  For details see the pdf files referenced above. Examples:
    cp -v /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/bwhpc-examples/{ammonia.xyz,methane.xyz,calculate.inp} ./
    calculate  -l calculate.inp -m AM1-COSMO > calculate.out  2>&1
    thermocalc -l calculate.inp -m ta        > thermocalc.out 2>&1

Using HUGE binaries of turbomole:
-------------------------------------------------------------------------------
* In addition a '*_huge' variant of most Turbomole commands is
  available after loading this module. This variant is intended for
  very huge calculations (e.g. when 'define' complains
  about 'limits are too low'). For details on '_huge' limits see
    /opt/bwhpc/common/chem/turbomole/6.6_tmolex40/documentation/README_howto_use_huge

Selecting a different Turbomole version (e.g. a parallel version):
-------------------------------------------------------------------------------
* By default
     module load chem/turbomole
  loads the serial version of Turbomole _without_ TmoleX support and
     module load chem/tmolex
  loads the serial version of Turbomole _with_ TmoleX support.
* If you want to use the shared memory parallel SMP version, you have to execute
     export PARA_ARCH=SMP
  _before_ calling the 'module load' command given above. In case of the message
  passing inferface (MPI) version, you have to execute
     export PARA_ARCH=MPI
  instead. The best PARA_ARCH depends on the Turbomole command you are planning
  to use. Please benchmark your jobs to find the optimal PARA_ARCH and PARNODES
  values before submitting mass jobs. Please read chapter 6.11 'Parallel runs'
  in 'Tutorial_6-6.pdf' and chapter 3.2 'Parallel Runs' in 'Documentation.pdf' 
  for a list of binaries and their parallelization.
* You can control the number of threads via environment variable PARNODES, e.g.
     export PARNODES=2
  You should set PARNODES in accordance with the number of requested
  cores in the queueing system (for details see pbs example script above).

Toubleshooting (possible problems and solutions):
-------------------------------------------------------------------------------
* If you did login via ssh, X11 forwarding must be enabled for TmoleX.
  E.g. login to current host via command 'ssh -X -l kn_pop235844 uc1n997'.
* Please be aware that TmoleX via X-forwarding is very slow.
  But you can download your own private copy of TmoleX (see above).
* Please load a newer Python and NumPy module if you see error messages like
    Minimum requirement of MoleControl is Python Version 2.6.2 final
  while using MoleControle (command 'mole').

Further software usage hints and conditions:
-------------------------------------------------------------------------------
* Please cite Turbomole in publications according to Turbomole documentation.
* The variables and commands are available after loading 'chem/turbomole/6.6'.
* In case of problems, please contact 'robert.barthel@kit.edu'.
 

---