Difference between revisions of "Environment Modules"

From bwHPC Wiki
Jump to: navigation, search
(Software job examples)
Line 12: Line 12:
 
An extensive online documentation is available on the [https://modules.readthedocs.io/en/latest/index.html Environment Modules Website].
 
An extensive online documentation is available on the [https://modules.readthedocs.io/en/latest/index.html Environment Modules Website].
 
<br>
 
<br>
= Usage =
+
= Basic Usage =
   
 
== Module categories, versions and defaults ==
 
== Module categories, versions and defaults ==
Line 26: Line 26:
   
 
Currently all bwHPC software packages are assigned to the following ''Module'' categories:
 
Currently all bwHPC software packages are assigned to the following ''Module'' categories:
  +
<!-- add wiki category for each of those, possibly just as a link -->
 
  +
<!-- -->
* [[:Category:Biology_software|bio]]
 
* [[:Category:Engineering_software|cae]]
+
[[:Category:Biology_software|bio]]
* [[:Category:Chemistry_software|chem]]
+
[[:Category:Engineering_software|cae]]
* [[:Category:Compiler_software|compiler]]
+
[[:Category:Chemistry_software|chem]]
* [[:Category:Debugger_software|devel]]
+
[[:Category:Compiler_software|compiler]]
* [[:Category:Mathematics_software|math]]
+
[[:Category:Debugger_software|devel]]
  +
[[:Category:Mathematics_software|math]]
* mpi
 
  +
mpi
* [[:Category:Numerical libraries|numlib]]
 
* [[:Category:Physics software|phys]]
+
[[:Category:Numerical libraries|numlib]]
* [[:Category:System software|system]]
+
[[:Category:Physics software|phys]]
* [[:Category:Visualization|vis]]
+
[[:Category:System software|system]]
  +
[[:Category:Visualization|vis]]
   
 
== Display and search available Modules ==
 
== Display and search available Modules ==
Line 50: Line 51:
 
$ module avail compiler/gnu
 
$ module avail compiler/gnu
 
</pre>
 
</pre>
  +
  +
== module help ==
  +
A help message for a specific ''Module'' can be displayed with ''''module help category/softwarename/version''''.
  +
<br>
  +
The help message usually contains additional information about the software and points to the software website and documentation.
  +
<pre>
  +
$ module help compiler/gnu/9.2
  +
  +
----------- Module Specific Help for 'compiler/gnu/9.2' -----------
  +
  +
  +
This module provides the GNU compiler collection version 9.2.0 via commands
  +
[... rest of the output is omitted in the Wiki for clarity ...]
  +
</pre>
  +
   
 
== Loading Modules ==
 
== Loading Modules ==
Line 59: Line 75:
 
Loading a ''Module'' in this manner makes the software accessible ONLY in the current shell / terminal.
 
Loading a ''Module'' in this manner makes the software accessible ONLY in the current shell / terminal.
   
  +
=== Software job examples ===
 
  +
== Display your loaded Modules ==
  +
All ''Modules'' that are currently loaded for you can be displayed by the
  +
command ''''module list''''.
  +
<pre>
  +
$ module list
  +
No Modulefiles Currently Loaded.
  +
$ module load compiler/gnu/9.2
  +
$ module list
  +
Currently Loaded Modulefiles:
  +
1) compiler/gnu/9.2
  +
</pre>
  +
  +
== Software job examples ==
 
bwHPC provides example job scripts for most installed software modules.
 
bwHPC provides example job scripts for most installed software modules.
   
Line 71: Line 100:
 
----
 
----
 
[[Category:System software]][[Category:bwUniCluster|Environment Modules]][[Category:ForHLR Phase I|Environment Modules]]
 
[[Category:System software]][[Category:bwUniCluster|Environment Modules]][[Category:ForHLR Phase I|Environment Modules]]
  +
  +
  +
= Additional Usage Recommendations =
   
 
=== Loading conflicts ===
 
=== Loading conflicts ===
Line 140: Line 172:
 
<br>
 
<br>
   
== Display your loaded Modules ==
 
All ''Modules'' that are currently loaded for you can be displayed by the
 
command ''''module list''''.
 
<pre>
 
$ module list
 
No Modulefiles Currently Loaded.
 
$ module load compiler/gnu/9.2
 
$ module list
 
Currently Loaded Modulefiles:
 
1) compiler/gnu/9.2
 
</pre>
 
 
== Unloading Modules ==
 
== Unloading Modules ==
 
To unload or to remove a software ''Module'' execute:
 
To unload or to remove a software ''Module'' execute:
Line 181: Line 202:
 
$ module whatis compiler/gnu/9.2
 
$ module whatis compiler/gnu/9.2
 
compiler/gnu/9.2 : GNU compiler suite version 9.2.0 (gcc, g++, gfortran, gccgo)
 
compiler/gnu/9.2 : GNU compiler suite version 9.2.0 (gcc, g++, gfortran, gccgo)
</pre>
 
=== module help ===
 
A help message for a specific ''Module'' can be displayed with ''''module help category/softwarename/version''''.
 
<br>
 
The help message usually contains additional information about the software and points to the software website and documentation.
 
<pre>
 
$ module help compiler/gnu/9.2
 
 
----------- Module Specific Help for 'compiler/gnu/9.2' -----------
 
 
 
This module provides the GNU compiler collection version 9.2.0 via commands
 
[... rest of the output is omitted in the Wiki for clarity ...]
 
 
</pre>
 
</pre>

Revision as of 12:13, 23 November 2021

Software on the bwHPC Clusters is provided as Software Environment Modules, or short Modules.

Modules make it possible to have different versions of a software installed at a the same time. The complete environments for the software package, compilers and libraries and needed by this specific version is then loaded by a single command. This happens usually in the beginning of the jobscript.

1 Documentation

For an extensive help on how to use Environment Modules, i.e., the command module, execute 'module help' or use 'man modules' to show manpage for the manual command.

An extensive online documentation is available on the Environment Modules Website.

2 Basic Usage

2.1 Module categories, versions and defaults

The bwHPC clusters categorize Modules, each software can exist in different versions:

category/softwarename/version

For instance the Intel compiler X.Y belongs to the category of compilers, therefore the modulefile X.Y is placed under the category compiler and intel.
In case of multiple software versions, one version will be always defined as the default version. The Module of the default can be addressed by simply omitting the version number:

category/softwarename

Currently all bwHPC software packages are assigned to the following Module categories:

bio cae chem compiler devel math mpi numlib phys system vis

2.2 Display and search available Modules

Available Modules are modulefiles that can be loaded by the user. A Module must be loaded before it provides changes to your environment. You can display all available Modules on the system by executing:

$ module avail

You can selectively list software in one of those categories using, e.g. for the category "compiler", or just all versions of a certain module:

$ module avail compiler/
$ module avail compiler/gnu

2.3 module help

A help message for a specific Module can be displayed with 'module help category/softwarename/version'.
The help message usually contains additional information about the software and points to the software website and documentation.

$ module help compiler/gnu/9.2 

----------- Module Specific Help for 'compiler/gnu/9.2' -----------


This module provides the GNU compiler collection version 9.2.0 via commands
[... rest of the output is omitted in the Wiki for clarity ...]


2.4 Loading Modules

To load a software Module:

$ module load category/softwarename/version

Loading a Module in this manner makes the software accessible ONLY in the current shell / terminal.


2.5 Display your loaded Modules

All Modules that are currently loaded for you can be displayed by the command 'module list'.

$ module list
No Modulefiles Currently Loaded.
$ module load compiler/gnu/9.2
$ module list
Currently Loaded Modulefiles:
  1) compiler/gnu/9.2

2.6 Software job examples

bwHPC provides example job scripts for most installed software modules.

It uses a variable named $SWN_EXA_DIR for a Module called SWN. To copy the whole example folder to your $HOME directory, so you can edit those job examples.

To copy the entire job examples folder of software swn to your working directory, execute:

$ module load catogory/softwarename
$ cp -R $SWN_EXA_DIR ./software-examples


3 Additional Usage Recommendations

3.1 Loading conflicts

By default you can not load different versions of same software Module in same session. Loading for example Intel compiler version X while Intel compiler version Y is loaded results in error message as follows:

Module 'compiler/intel/X' conflicts with the currently loaded module(s) 'compiler/intel/Y'

The solution is unloading or switching Modules.

3.2 Showing the changes introduced by a Module

Loading a Module will change the environment of the current shell session. For instance the $PATH variable will be expanded by the software's binary directory. Other Module variables may even change the behavior of the current shell session or the software program(s) in a more drastic way.
Loaded Modules may also invoke an additional set of environment variables, which e.g. point to directories or destinations of documentation and examples. Their nomenclature is systematic:

Variable Pointing to
$SWN_HOME Root directory of the software package
$SWN_DOC_DIR Documentation
$SWN_EXA_DIR Examples
$SWN_BPR_URL URL of software's Wiki article
and many many more...  

with SWN being the place holder of the software Module name.
All the changes to the current shell session to be invoked by loading the Module can be reviewed using 'module show category/softwarename/version'.

$ module show compiler/gnu/9.2
-------------------------------------------------------------------
/opt/bwhpc/common/modulefiles/compiler/gnu/9.2:

setenv           GNU_VERSION 9.2.0 
setenv           GNU_HOME /opt/bwhpc/common/compiler/gnu/9.2.0 
setenv           GNU_BIN_DIR /opt/bwhpc/common/compiler/gnu/9.2.0/bin 
setenv           GNU_MAN_DIR /opt/bwhpc/common/compiler/gnu/9.2.0/man 
setenv           GNU_LIB_DIR /opt/bwhpc/common/compiler/gnu/9.2.0/lib64 
setenv           GNU_LIB_DIR2 /opt/bwhpc/common/compiler/gnu/9.2.0/lib 
prepend-path     PATH /opt/bwhpc/common/compiler/gnu/9.2.0/bin 
prepend-path     MANPATH /opt/bwhpc/common/compiler/gnu/9.2.0/man 
prepend-path     LD_LIBRARY_PATH /opt/bwhpc/common/compiler/gnu/9.2.0/lib64:/opt/bwhpc/common/compiler/gnu/9.2.0/lib 
setenv           CC gcc 
setenv           CXX g++ 
setenv           F77 gfortran 
setenv           FC gfortran 
setenv           F90 gfortran 
module-whatis    GNU compiler suite version 9.2.0 (gcc, g++, gfortran, gccgo) 
conflict         compiler/gnu 
-------------------------------------------------------------------

3.3 Modules depending on Modules

Some program Modules depend on libraries to be loaded to the user environment. Therefore the corresponding Modules of the software must be loaded together with the Modules of the libraries.
By default such software Modules try to load required Modules and corresponding versions automatically. However, automatic loading might fail if a different version of that required Module is already loaded (cf. Loading conflicts).

3.4 Unloading Modules

To unload or to remove a software Module execute:

$ module unload category/softwarename/version

or

$ module remove category/softwarename/version

3.4.1 Unloading all loaded modules

In order to remove all previously loaded software modules from your environment issue the command 'module purge'.

$ module list
Currently Loaded Modulefiles:
  1) devel/gdb/7.7
  2) compiler/intel/14.0
  3) mpi/openmpi/1.8-intel-14.0(default)
$
$ module purge
$ module list
No Modulefiles Currently Loaded.
$ 

3.5 Other Module commands

3.5.1 module whatis

A short description for a specific Module can be displayed with 'module whatis category/softwarename/version'

$ module whatis compiler/gnu/9.2 
compiler/gnu/9.2     : GNU compiler suite version 9.2.0 (gcc, g++, gfortran, gccgo)