Difference between revisions of "Environment Modules"

From bwHPC Wiki
Jump to: navigation, search
(bwHPC Cluster Information System (CIS): existiert nicht mehr)
(Überarbeitung im Rahmen der bwHPC-Wiki Überarbeitung)
Line 1: Line 1:
{| width=650px class="wikitable"
 
|-
 
! Description !! Content
 
|-
 
| module load
 
| category/name | category/name/version (optional)
 
|-
 
| Links
 
| [http://modules.sourceforge.net/ Environment Modules Project] | [https://sourceforge.net/projects/modules/ Environment Modules]
 
|-
 
| License
 
| [http://www.gnu.org/licenses/old-licenses/gpl-2.0.html GNU General Public License]
 
|}
 
<br>
 
 
= Introduction =
 
= Introduction =
 
'''Environment Modules''', or short '''Modules''' are the means by which most of the installed scientific software is provided on the bwHPC clusters.
 
'''Environment Modules''', or short '''Modules''' are the means by which most of the installed scientific software is provided on the bwHPC clusters.
 
<br>
 
<br>
The use of different compilers, libraries and software packages requires users to set up a specific session environment suited for the program they want to run. The bwHPC clusters provide users with the possibility to load and unload complete environments for compilers, libraries and software packages by a single command.
+
The use of different compilers, libraries and software packages requires users to set up a specific session environment suited for the program they want to run. The bwHPC clusters provide users with the possibility to load and unload complete environments for compilers, libraries and software packages by a single command. This happens usually in the beginning of the jobscript.
 
<br>
 
<br>
  +
= Documentation =
  +
  +
For an extensive help on how to use Environment Modules, i.e., the command '''module''',
  +
execute ''''module help'''' or use ''''man modules'''' to show manpage for the '''manual''' command.
  +
  +
An extensive online documentation is available on the [https://modules.readthedocs.io/en/latest/index.html Environment Modules Website].
 
<br>
 
<br>
= Description =
 
The Environment ''Modules'' package enables dynamic modification of your environment by the
 
use of so-called ''modulefiles''. A ''modulefile'' contains information to configure the shell
 
for a program/software . Typically, a modulefile contains instructions that alter or set shell
 
environment variables, such as PATH and MANPATH, to enable access to various installed
 
software.
 
<br>
 
One of the key features of using the Environment ''Modules'' software is to allow multiple versions of the same software to be used in your environment in a controlled manner.
 
For example, two different versions of the Intel C compiler can be installed on the system at the same time - the version used is based upon which Intel C compiler modulefile is loaded.
 
<br>
 
The software stack of bwHPC clusters provides a number of modulefiles. You can also
 
create your own modulefiles. ''Modulefiles'' may be shared by many users on a system, and
 
users may have their own collection of modulefiles to supplement or replace the shared
 
modulefiles.
 
<br>
 
A modulefile does not provide configuration of your environment until it is explicitly loaded,
 
i.e., the specific modulefile for a software product or application must be loaded in your environment before the configuration information in the modulefile is effective.
 
<br>
 
For instance loading the default Intel C and Fortran compiler you must execute
 
''''module load compiler/intel''''.
 
<br>
 
<pre>
 
$ module load compiler/intel
 
$ module list
 
Currently Loaded Modulefiles:
 
1) compiler/pgi/12.10(default) 2) compiler/intel/15.0(default)
 
$ : Display all Intel related environments now
 
$ env | grep INTEL
 
INTEL_LICENSE_FILE=/opt/bwhpc/common/compiler/intel/compxe.2015.3.187/composer_xe_2015.3.187/licenses
 
INTEL_LIB_MICMPI=/opt/bwhpc/common/compiler/intel/compxe.2015.3.187/composer_xe_2015.3.187/mpirt/lib/mic
 
INTEL_HOME=/opt/bwhpc/common/compiler/intel/compxe.2015.3.187/composer_xe_2015.3.187
 
INTEL_VERSION=15.0.3
 
INTEL_MAN_DIR=/opt/bwhpc/common/compiler/intel/compxe.2015.3.187/composer_xe_2015.3.187/man/en_US
 
INTEL_INC_DIR=/opt/bwhpc/common/compiler/intel/compxe.2015.3.187/composer_xe_2015.3.187/compiler/include
 
INTEL_BIN_DIR=/opt/bwhpc/common/compiler/intel/compxe.2015.3.187/composer_xe_2015.3.187/bin/intel64
 
INTEL_DOC_DIR=/opt/bwhpc/common/compiler/intel/compxe.2015.3.187/composer_xe_2015.3.187/Documentation/en_US
 
INTEL_LIB_DIR=/opt/bwhpc/common/compiler/intel/compxe.2015.3.187/composer_xe_2015.3.187/compiler/lib/intel64
 
INTEL_LIB_MIC=/opt/bwhpc/common/compiler/intel/compxe.2015.3.187/composer_xe_2015.3.187/compiler/lib/mic
 
INTEL_PYTHONHOME=/opt/bwhpc/common/compiler/intel/compxe.2015.3.187/composer_xe_2015.3.187/debugger/python/intel64
 
</pre>
 
<br>
 
 
 
= Usage =
 
= Usage =
== Documentation ==
 
For help on how to use ''Modules'' software, i.e., the command '''module''',
 
execute ''''module help''''.
 
<pre>
 
$ module help
 
Modules Release 3.2.10 2012-12-21 (Copyright GNU GPL v2 1991):
 
   
Usage: module [ switches ] [ subcommand ] [subcommand-args ]
 
 
Switches:
 
-H|--help this usage info
 
-V|--version modules version & configuration options
 
-f|--force force active dependency resolution
 
-t|--terse terse format avail and list format
 
-l|--long long format avail and list format
 
-h|--human readable format avail and list format
 
-v|--verbose enable verbose messages
 
-s|--silent disable verbose messages
 
-c|--create create caches for avail and apropos
 
-i|--icase case insensitive
 
-u|--userlvl <lvl> set user level to (nov[ice],exp[ert],adv[anced])
 
Available SubCommands and Args:
 
+ add|load modulefile [modulefile ...]
 
+ rm|unload modulefile [modulefile ...]
 
+ switch|swap [modulefile1] modulefile2
 
+ display|show modulefile [modulefile ...]
 
+ avail [modulefile [modulefile ...]]
 
+ use [-a|--append] dir [dir ...]
 
+ unuse dir [dir ...]
 
+ update
 
+ refresh
 
+ purge
 
+ list
 
+ clear
 
+ help [modulefile [modulefile ...]]
 
+ whatis [modulefile [modulefile ...]]
 
+ apropos|keyword string
 
+ initadd modulefile [modulefile ...]
 
+ initprepend modulefile [modulefile ...]
 
+ initrm modulefile [modulefile ...]
 
+ initswitch modulefile1 modulefile2
 
+ initlist
 
+ initclear
 
</pre>
 
or ''''man module''''.
 
<pre>
 
MODULE(1) Modules package MODULE(1)
 
 
NAME
 
module - command interface to the Modules package
 
 
SYNOPSIS
 
module [ switches ] [ sub-command ] [ sub-command-args ]
 
 
DESCRIPTION
 
module is a user interface to the Modules package. The Modules package
 
provides for the dynamic modification of the user's environment via
 
modulefiles.
 
 
Each modulefile contains the information needed to configure the shel
 
[...]
 
</pre>
 
For help on particular version of ''Module'', e.g. Intel compiler version X.Y, execute
 
''''module help compiler/intel''''.
 
<pre>
 
$ module help compiler/intel
 
----------- Module Specific Help for 'compiler/intel/15.0' --------
 
This module provides the Intel(R) compiler suite version 15.0.3 via
 
commands 'icc', 'icpc' and 'ifort' (version 15.0.3), the debugger 'gdb-ia' (version
 
7.8.3) as well as the Intel(R) Threading Building Blocks TBB (version 4.3.5)
 
and the Integrated Performance Primitives IPP libraries (version 8.2.2)
 
(for details see also 'http://software.intel.com/en-us/intel-compilers/').
 
 
The related Math Kernel Library MKL module is 'numlib/mkl/11.2.3'.
 
The related Intel MPI module is 'mpi/impi/5.0.3-intel-15.0'.
 
The Intel 'icpc' should work well with GNU compiler version 4.4 to 4.8.
 
Before using TBB or IPP setup the corresponding environment, e.g. for 64bit+bash
 
source $INTEL_HOME/tbb/bin/tbbvars.sh intel64
 
source $INTEL_HOME/ipp/bin/ippvars.sh intel64
 
 
Commands:
 
icc # Intel(R) C compiler
 
icpc # Intel(R) C++ compiler
 
ifort # Intel(R) Fortran compiler
 
gdb-ia # Intel version of GNU debugger
 
# idb is not available anymore in Intel compiler suite 2015.
 
 
Local documentation:
 
Man pages: man icc; man icpc; man ifort; man gdb-ia
 
firefox $INTEL_DOC_DIR/beginusing_lc.htm
 
firefox $INTEL_DOC_DIR/beginusing_lf.htm
 
The html-pages are very detailed and cover TBB and IPP as well as MKL.
 
 
For some Intel(R) compiler option examples, hints on how to compile 32bit code
 
and solutions for less common problems see the tips and troubleshooting doc:
 
$INTEL_DOC_DIR/intel-compiler-tips-and-troubleshooting.txt
 
 
For details on library and include dirs please call
 
module show compiler/intel/15.0
 
[...]
 
</pre>
 
=== Online Documentation ===
 
[https://sourceforge.net/p/modules/wiki/FAQ/ Frequently Asked Questions (FAQ)]
 
 
== Display all available Modules ==
 
Available ''Module'' are modulefiles that can be loaded by the user. A ''Module'' must be loaded before it provides changes to your environment, as described in the introduction to this
 
section. You can display all available ''Modules'' on the system by executing:
 
<pre>
 
$ module avail
 
</pre>
 
The short form the command is:
 
<pre>
 
$ module av
 
</pre>
 
Available ''Modules'' can be also displayed in different modes, such as
 
* each ''Module'' per one line
 
<pre>
 
$ module -t avail
 
</pre>
 
* long
 
<pre>
 
$ module -l avail
 
</pre>
 
 
== Module categories, versions and defaults ==
 
== Module categories, versions and defaults ==
 
The bwHPC clusters (such as [[bwUniCluster]]) traditionally provide a large variety of
 
The bwHPC clusters (such as [[bwUniCluster]]) traditionally provide a large variety of
software and software versions. Therefore ''Module'' are divided in category folders
+
software and software versions. Therefore ''Modules'' are divided in category folders
 
containing subfolders of modulefiles again containing modulefile versions, and must be addressed
 
containing subfolders of modulefiles again containing modulefile versions, and must be addressed
 
as follows:
 
as follows:
Line 197: Line 25:
 
version. The ''Module'' of the default can be addressed by simply omitting the version number:
 
version. The ''Module'' of the default can be addressed by simply omitting the version number:
 
category/softwarename
 
category/softwarename
  +
== Finding software Modules ==
 
 
Currently all bwHPC software packages are assigned to the following ''Module'' categories:
 
Currently all bwHPC software packages are assigned to the following ''Module'' categories:
 
<!-- add wiki category for each of those, possibly just as a link -->
 
<!-- add wiki category for each of those, possibly just as a link -->
Line 211: Line 39:
 
* [[:Category:System software|system]]
 
* [[:Category:System software|system]]
 
* [[:Category:Visualization|vis]]
 
* [[:Category:Visualization|vis]]
  +
  +
== Display and search available Modules ==
  +
Available ''Modules'' are modulefiles that can be loaded by the user. A ''Module'' must be loaded before it provides changes to your environment. You can display all available ''Modules'' on the system by executing:
  +
<pre>
  +
$ module available
  +
</pre>
  +
The short form the command is:
  +
<pre>
  +
$ module av
  +
</pre>
  +
 
You can selectively list software in one of those categories using, e.g. for the category "compiler"
 
You can selectively list software in one of those categories using, e.g. for the category "compiler"
 
<pre>
 
<pre>
 
$ module avail compiler/
 
$ module avail compiler/
 
</pre>
 
</pre>
  +
This would list all available versions of the GCC compiler suite
Searches are looking for a substring starting at the begin of the name, so this would list all software in categories starting with a "c"
 
<pre>
 
$ module avail c
 
</pre>
 
while this would find nothing
 
 
<pre>
 
<pre>
$ module avail hem
+
$ module avail compiler/gnu
 
</pre>
 
</pre>
   
Line 270: Line 105:
 
All the changes to the current shell session to be invoked by loading the ''Module'' can be reviewed using ''''module show category/softwarename/version''''.
 
All the changes to the current shell session to be invoked by loading the ''Module'' can be reviewed using ''''module show category/softwarename/version''''.
 
<br>
 
<br>
<u>Example (Intel compiler)</u>
 
 
<pre>
 
<pre>
$ module show compiler/intel/14.0
+
$ module show compiler/gnu/9.2
 
-------------------------------------------------------------------
 
-------------------------------------------------------------------
/opt/bwhpc/common/modulefiles/compiler/intel/14.0:
+
/opt/bwhpc/common/modulefiles/compiler/gnu/9.2:
  +
module-whatis Intel(R) compiler suite (icc, icpc, ifort), debugger (idb), IPP and TBB ver 14.0.4
 
setenv INTEL_VERSION 14.0.4
+
setenv GNU_VERSION 9.2.0
setenv INTEL_HOME /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211
+
setenv GNU_HOME /opt/bwhpc/common/compiler/gnu/9.2.0
setenv INTEL_BIN_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/bin/intel64
+
setenv GNU_BIN_DIR /opt/bwhpc/common/compiler/gnu/9.2.0/bin
setenv INTEL_LIB_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/lib/intel64
+
setenv GNU_MAN_DIR /opt/bwhpc/common/compiler/gnu/9.2.0/man
setenv INTEL_LIB_MIC /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/lib/mic
+
setenv GNU_LIB_DIR /opt/bwhpc/common/compiler/gnu/9.2.0/lib64
setenv INTEL_INC_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/include
+
setenv GNU_LIB_DIR2 /opt/bwhpc/common/compiler/gnu/9.2.0/lib
setenv INTEL_MAN_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/man/en_US
+
prepend-path PATH /opt/bwhpc/common/compiler/gnu/9.2.0/bin
setenv INTEL_DOC_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/Documentation/en_US
+
prepend-path MANPATH /opt/bwhpc/common/compiler/gnu/9.2.0/man
  +
prepend-path LD_LIBRARY_PATH /opt/bwhpc/common/compiler/gnu/9.2.0/lib64:/opt/bwhpc/common/compiler/gnu/9.2.0/lib
setenv ICC_VERSION 14.0.4
 
  +
setenv CC gcc
setenv ICC_HOME /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211
 
  +
setenv CXX g++
setenv ICC_BIN_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/bin/intel64
 
  +
setenv F77 gfortran
setenv ICC_LIB_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/lib/intel64
 
  +
setenv FC gfortran
setenv ICC_INC_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/include
 
  +
setenv F90 gfortran
setenv ICC_MAN_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/man/en_US
 
  +
module-whatis GNU compiler suite version 9.2.0 (gcc, g++, gfortran, gccgo)
setenv ICC_DOC_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/Documentation/en_US
 
  +
conflict compiler/gnu
setenv IFORT_VERSION 14.0.4
 
setenv IFORT_HOME /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211
 
setenv IFORT_BIN_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/bin/intel64
 
setenv IFORT_LIB_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/lib/intel64
 
setenv IFORT_INC_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/include
 
setenv IFORT_MAN_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/man/en_US
 
setenv IFORT_DOC_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/Documentation/en_US
 
setenv IDB_VERSION 14.0.4
 
setenv IDB_HOME /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211
 
setenv IDB_LIB_DIR /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/debugger/lib/intel64
 
setenv LANGUAGE_TERRITORY en_US
 
prepend-path PATH /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/bin/intel64
 
prepend-path LD_LIBRARY_PATH /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/lib/intel64
 
prepend-path LD_RUN_PATH /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/lib/intel64
 
prepend-path MIC_LD_LIBRARY_PATH /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/lib/mic
 
prepend-path LIBRARY_PATH /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/lib/intel64
 
prepend-path MANPATH /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/man/en_US
 
prepend-path NLSPATH /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/compiler/lib/intel64/locale/%l_%t/%N
 
prepend-path LD_LIBRARY_PATH /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/debugger/lib/intel64
 
prepend-path NLSPATH /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/debugger/intel64/locale/%l_%t/%N
 
prepend-path INTEL_LICENSE_FILE /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/composer_xe_2013_sp1.4.211/licenses
 
setenv IDB_JAVA_ARGUMENTS -Xms512m -Xmx1024m
 
setenv CC icc
 
setenv CXX icpc
 
setenv F77 ifort
 
setenv FC ifort
 
setenv F90 ifort
 
setenv TEST_MODULE_SCRIPT /opt/bwhpc/common/compiler/intel/compxe.2013.sp1.4.211/install-doc/test-compiler-intel.sh
 
setenv TEST_MODULE_NAME compiler/intel/14.0
 
conflict compiler/intel
 
conflict compiler/gnu/4.9
 
 
-------------------------------------------------------------------
 
-------------------------------------------------------------------
 
</pre>
 
</pre>
<font color=red>'module show' does '''not''' load the ''Module''!</font>
 
   
 
=== Modules depending on Modules ===
 
=== Modules depending on Modules ===
Line 334: Line 137:
 
is already loaded (cf. [[#Loading conflicts|Loading conflicts]]).
 
is already loaded (cf. [[#Loading conflicts|Loading conflicts]]).
 
<br>
 
<br>
  +
== Display your loaded Modules ==
  +
All ''Modules'' that are currently loaded for you can be displayed by the
  +
command ''''module list''''.
  +
<pre>
  +
$ module list
  +
No Modulefiles Currently Loaded.
  +
$ module load compiler/gnu/9.2
  +
$ module list
  +
Currently Loaded Modulefiles:
  +
1) compiler/gnu/9.2
  +
</pre>
 
== Unloading Modules ==
 
== Unloading Modules ==
 
To unload or to remove a software ''Module'' execute:
 
To unload or to remove a software ''Module'' execute:
Line 344: Line 158:
 
</pre>
 
</pre>
 
=== Unloading all loaded modules ===
 
=== Unloading all loaded modules ===
==== Purge ====
 
Unloading a ''Module'' that has been loaded by default makes it inactive for the current session only - it will be reloaded the next time you log in.
 
<br>
 
 
In order to remove all previously loaded software modules from your environment issue the command 'module purge'.
 
In order to remove all previously loaded software modules from your environment issue the command 'module purge'.
 
<br>
 
<br>
<u>Example</u>
 
 
<pre>
 
<pre>
 
$ module list
 
$ module list
Line 362: Line 172:
 
$
 
$
 
</pre>
 
</pre>
  +
== Other Module commands ==
<font color>Beware!</font>
 
  +
=== module whatis ===
  +
A short description for a specific ''Module'' can be displayed with ''''module whatis category/softwarename/version''''
  +
<pre>
  +
$ module whatis compiler/gnu/9.2
  +
compiler/gnu/9.2 : GNU compiler suite version 9.2.0 (gcc, g++, gfortran, gccgo)
  +
</pre>
  +
=== module help ===
  +
A help message for a specific ''Module'' can be displayed with ''''module help category/softwarename/version''''.
 
<br>
 
<br>
  +
The help message usually contains additional information about the software and points to the software website and documentation.
'module purge' is working without any further inquiry.
 
  +
<pre>
  +
$ module help compiler/gnu/9.2
   
  +
----------- Module Specific Help for 'compiler/gnu/9.2' -----------
==== Clear ====
 
  +
Use ''''module clear'''' and confirm with "'''y'''" to unload all loaded module, too.
 
  +
<br>
 
  +
This module provides the GNU compiler collection version 9.2.0 via commands
<u>Example</u>
 
  +
[... rest of the output is omitted in the Wiki for clarity ...]
<pre>
 
$ module list
 
Currently Loaded Modulefiles:
 
1) compiler/intel/14.0
 
2) mpi/openmpi/1.8-intel-14.0(default)
 
3) devel/gdb/7.7
 
$
 
$ module clear
 
Are you sure you want to clear all loaded modules!? [n] y
 
$
 
$ module list
 
No Modulefiles Currently Loaded.
 
 
</pre>
 
</pre>
 
== Display your loaded Modules ==
 
All ''Modules'' that are currently loaded for you can be displayed by the
 
command ''''module list''''. [[#Purge|See example above]].
 
<br>
 
Note: You only have to load further ''Modules'', if you want to use additional software
 
packages or to change the version of an already loaded software.
 
<br>
 
<br>
 
   
 
= Software job examples =
 
= Software job examples =
Line 402: Line 202:
 
$ cp -R $SWN_EXA_DIR .
 
$ cp -R $SWN_EXA_DIR .
 
</pre>
 
</pre>
<br>
 
= How do Modules work? =
 
The default shell on the bwHPC clusters is bash, so explanations and examples will be shown for bash. In general, programs cannot modify the environment of the shell they are being run from, so how can the module command do exactly that?
 
<br>
 
The module command is not a program, but a bash-function.
 
You can view its content using
 
<pre>
 
$ type module
 
</pre>
 
and you will get a result like this:
 
<pre>
 
$ type module
 
module is a function
 
module ()
 
{
 
eval `/usr/bin/modulecmd bash $*`
 
}
 
</pre>
 
In this function, modulecmd is called. Its output to stdout is then executed inside your current shell using the bash-internal ''eval'' command. As a consequence, all output that you see from the module is transmitted via stderr (output handle 2) or in some cases even stdin (output handle 0).
 
<br>
 
<br>
 
 
----
 
----
 
[[Category:System software]][[Category:bwUniCluster|Environment Modules]][[Category:ForHLR Phase I|Environment Modules]]
 
[[Category:System software]][[Category:bwUniCluster|Environment Modules]][[Category:ForHLR Phase I|Environment Modules]]

Revision as of 14:16, 9 November 2021

1 Introduction

Environment Modules, or short Modules are the means by which most of the installed scientific software is provided on the bwHPC clusters.
The use of different compilers, libraries and software packages requires users to set up a specific session environment suited for the program they want to run. The bwHPC clusters provide users with the possibility to load and unload complete environments for compilers, libraries and software packages by a single command. This happens usually in the beginning of the jobscript.

2 Documentation

For an extensive help on how to use Environment Modules, i.e., the command module, execute 'module help' or use 'man modules' to show manpage for the manual command.

An extensive online documentation is available on the Environment Modules Website.

3 Usage

3.1 Module categories, versions and defaults

The bwHPC clusters (such as bwUniCluster) traditionally provide a large variety of software and software versions. Therefore Modules are divided in category folders containing subfolders of modulefiles again containing modulefile versions, and must be addressed as follows:

category/softwarename/version

For instance the Intel compiler X.Y belongs to the category of compilers, therefore the modulefile X.Y is placed under the category compiler and intel.
In case of multiple software versions, one version will be always defined as the default version. The Module of the default can be addressed by simply omitting the version number:

category/softwarename

Currently all bwHPC software packages are assigned to the following Module categories:

3.2 Display and search available Modules

Available Modules are modulefiles that can be loaded by the user. A Module must be loaded before it provides changes to your environment. You can display all available Modules on the system by executing:

$ module available

The short form the command is:

$ module av

You can selectively list software in one of those categories using, e.g. for the category "compiler"

$ module avail compiler/

This would list all available versions of the GCC compiler suite

$ module avail compiler/gnu

3.3 Loading Modules

You can load a Module software in to your environment to enable easier access to software that you want to use by executing:

$ module load category/softwarename/version

or

$ module add category/softwarename/version

Loading a Module in this manner affects ONLY your environment for the current session.

3.3.1 Loading conflicts

By default you can not load different versions of same software Module in same session. Loading for example Intel compiler version X while Intel compiler version Y is loaded results in error message as follows:

Module 'compiler/intel/X' conflicts with the currently loaded module(s) 'compiler/intel/Y'

The solution is unloading or switching Modules.

3.3.2 Showing the changes introduced by a Module

Loading a Module will change the environment of the current shell session. For instance the $PATH variable will be expanded by the software's binary directory. Other Module variables may even change the behavior of the current shell session or the software program(s) in a more drastic way.
Loaded Modules may also invoke an additional set of environment variables, which e.g. point to directories or destinations of documentation and examples. Their nomenclature is systematic:

Variable Pointing to
$SWN_HOME Root directory of the software package
$SWN_DOC_DIR Documentation
$SWN_EXA_DIR Examples
$SWN_BPR_URL URL of software's Wiki article
and many many more...  

with SWN being the place holder of the software Module name.
All the changes to the current shell session to be invoked by loading the Module can be reviewed using 'module show category/softwarename/version'.

$ module show compiler/gnu/9.2
-------------------------------------------------------------------
/opt/bwhpc/common/modulefiles/compiler/gnu/9.2:

setenv           GNU_VERSION 9.2.0 
setenv           GNU_HOME /opt/bwhpc/common/compiler/gnu/9.2.0 
setenv           GNU_BIN_DIR /opt/bwhpc/common/compiler/gnu/9.2.0/bin 
setenv           GNU_MAN_DIR /opt/bwhpc/common/compiler/gnu/9.2.0/man 
setenv           GNU_LIB_DIR /opt/bwhpc/common/compiler/gnu/9.2.0/lib64 
setenv           GNU_LIB_DIR2 /opt/bwhpc/common/compiler/gnu/9.2.0/lib 
prepend-path     PATH /opt/bwhpc/common/compiler/gnu/9.2.0/bin 
prepend-path     MANPATH /opt/bwhpc/common/compiler/gnu/9.2.0/man 
prepend-path     LD_LIBRARY_PATH /opt/bwhpc/common/compiler/gnu/9.2.0/lib64:/opt/bwhpc/common/compiler/gnu/9.2.0/lib 
setenv           CC gcc 
setenv           CXX g++ 
setenv           F77 gfortran 
setenv           FC gfortran 
setenv           F90 gfortran 
module-whatis    GNU compiler suite version 9.2.0 (gcc, g++, gfortran, gccgo) 
conflict         compiler/gnu 
-------------------------------------------------------------------

3.3.3 Modules depending on Modules

Some program Modules depend on libraries to be loaded to the user environment. Therefore the corresponding Modules of the software must be loaded together with the Modules of the libraries.
By default such software Modules try to load required Modules and corresponding versions automatically. However, automatic loading might fail if a different version of that required Module is already loaded (cf. Loading conflicts).

3.4 Display your loaded Modules

All Modules that are currently loaded for you can be displayed by the command 'module list'.

$ module list
No Modulefiles Currently Loaded.
$ module load compiler/gnu/9.2
$ module list
Currently Loaded Modulefiles:
  1) compiler/gnu/9.2

3.5 Unloading Modules

To unload or to remove a software Module execute:

$ module unload category/softwarename/version

or

$ module remove category/softwarename/version

3.5.1 Unloading all loaded modules

In order to remove all previously loaded software modules from your environment issue the command 'module purge'.

$ module list
Currently Loaded Modulefiles:
  1) devel/gdb/7.7
  2) compiler/intel/14.0
  3) mpi/openmpi/1.8-intel-14.0(default)
$
$ module purge
$ module list
No Modulefiles Currently Loaded.
$ 

3.6 Other Module commands

3.6.1 module whatis

A short description for a specific Module can be displayed with 'module whatis category/softwarename/version'

$ module whatis compiler/gnu/9.2 
compiler/gnu/9.2     : GNU compiler suite version 9.2.0 (gcc, g++, gfortran, gccgo)

3.6.2 module help

A help message for a specific Module can be displayed with 'module help category/softwarename/version'.
The help message usually contains additional information about the software and points to the software website and documentation.

$ module help compiler/gnu/9.2 

----------- Module Specific Help for 'compiler/gnu/9.2' -----------


This module provides the GNU compiler collection version 9.2.0 via commands
[... rest of the output is omitted in the Wiki for clarity ...]

4 Software job examples

The Modules installed on bwHPC systems provide job examples to help you get started using the software or submitting jobs with this software. Examples can be found via a convenient variable $SWN_EXA_DIR (for a Module called SWN). It is advisable to copy the whole example folder to your $HOME directory, so you can edit those job examples.
For copying the entire job examples folder of software swn to your working directory, execute:

$ module load catogory/softwarename
$ cp -R $SWN_EXA_DIR .