BwUniCluster3.0/Software Modules - Revision history

A Flachmueller: /* Build individual EasyBuild packages as a user */

2026-05-11T09:51:05Z

Build individual EasyBuild packages as a user

@@ Line 385: / Line 385: @@
 EasyBuild can also be used to compile your own or niche software, based on the available EasyBuild files by creating an independent EasyBuild software and module tree. The bwUnicluster 3.0 EasyBuild tree can be used as a base to build additional software in your HOME directory.
 We assume the following
@@ Line 412: / Line 405: @@
 <pre class="example">source /opt/bwhpc/common/etc/easybuild/enable_eb_modules
 </pre>
 * add local module path

A Flachmueller: /* Build individual EasyBuild packages as a user */

2026-05-11T09:29:54Z

Build individual EasyBuild packages as a user

← Older revision		Revision as of 11:29, 11 May 2026
Line 389:		Line 389:
	<pre class="example">module spider EasyBuild</pre>		<pre class="example">module spider EasyBuild</pre>

	You have to load ~~the~~ EasyBuild module:		You have to load an EasyBuild module, e.g.:

	<pre class="example">module load EasyBuild/5.3.0</pre>		<pre class="example">module load EasyBuild/5.3.0</pre>

A Flachmueller: /* Build individual EasyBuild packages as a user */

2026-05-11T09:29:30Z

Build individual EasyBuild packages as a user

← Older revision		Revision as of 11:29, 11 May 2026
Line 384:		Line 384:

	EasyBuild can also be used to compile your own or niche software, based on the available EasyBuild files by creating an independent EasyBuild software and module tree. The bwUnicluster 3.0 EasyBuild tree can be used as a base to build additional software in your HOME directory.		EasyBuild can also be used to compile your own or niche software, based on the available EasyBuild files by creating an independent EasyBuild software and module tree. The bwUnicluster 3.0 EasyBuild tree can be used as a base to build additional software in your HOME directory.

			Check provided EasyBuild installations:

			<pre class="example">module spider EasyBuild</pre>

			You have to load the EasyBuild module:

			<pre class="example">module load EasyBuild/5.3.0</pre>

	We assume the following		We assume the following

S Braun: /* EasyBuild modules */

2025-03-25T10:23:27Z

EasyBuild modules

← Older revision		Revision as of 12:23, 25 March 2025
Line 400:		Line 400:
	allow-modules-tool-mismatch=True		allow-modules-tool-mismatch=True
	</pre>		</pre>
	* add system-wide ~~easybuild~~ module path		* add system-wide EasyBuild module path

	<pre class="example">source /opt/bwhpc/common/etc/easybuild/enable_eb_modules		<pre class="example">source /opt/bwhpc/common/etc/easybuild/enable_eb_modules

S Braun: /* Build individual EasyBuild packages as a user */

2025-03-25T10:20:57Z

Build individual EasyBuild packages as a user

← Older revision		Revision as of 12:20, 25 March 2025
Line 409:		Line 409:
	module use ~/eb/modules/all		module use ~/eb/modules/all
	</pre>		</pre>
	* find or create ~~easybuild~~ file, you can use any of the ~~easybild~~		* find or create EasyBuild file, you can use any of the EasyBuild

	<pre class="example">ls ~/eb/easybuild-easyconfigs/easybuild/easyconfigs// \| grep eb$		<pre class="example">ls ~/eb/easybuild-easyconfigs/easybuild/easyconfigs// \| grep eb$

S Braun: /* EasyBuild modules */

2025-03-25T10:19:45Z

EasyBuild modules

← Older revision		Revision as of 12:19, 25 March 2025
Line 355:		Line 355:
	= EasyBuild modules =		= EasyBuild modules =

	With bwUniCluster 3.0 we start an official beta-test to provide toolchains and scientific software built with [https://docs.easybuild.io/ EasyBuild]. EasyBuild is a well established and consistent way to provide scientific software at many HPC sites all over the world. Since the naming of ~~Easybuild~~ modules is consistent users can use modules with the same name at different sites. To make this possible, Easybuild software minimizes the resources used by the local operating system and provides most software independently. This makes scientific results more reproducible.		With bwUniCluster 3.0 we start an official beta-test to provide toolchains and scientific software built with [https://docs.easybuild.io/ EasyBuild]. EasyBuild is a well established and consistent way to provide scientific software at many HPC sites all over the world. Since the naming of EasyBuild modules is consistent users can use modules with the same name at different sites. To make this possible, Easybuild software minimizes the resources used by the local operating system and provides most software independently. This makes scientific results more reproducible.

	<span id="using-easybuild-modules"></span>		<span id="using-easybuild-modules"></span>
Line 376:		Line 376:
	== EasyBuild conventions ==		== EasyBuild conventions ==

	In theory one could build every software against every library and every compiler. This would lead to an unmaintainable amount of combinations. For this reason the EasyBuild team has decided to provide two so called toolchains a year for the most common compiler suits. Each toolchain defines one version of compiler, MPI library, math libraries and other packages as default for all software ~~build~~ aginst this toolchain. On bwUniCluster 3.0 we start with 3 ~~foss~~, 3 ~~intel~~ and one ~~nvhpc~~ toolchains. For a complete list of EasyBuild toolchains, please see [https://docs.easybuild.io/version-specific/toolchains/ the EasyBuild website]. Currently the 2023a, 2023b, and 2024a toolchains are available, with 2024a being the default one. Some software may not be available for the latest toolchain yet, but may be available for an older toolchain.		In theory one could build every software against every library and every compiler. This would lead to an unmaintainable amount of combinations. For this reason the EasyBuild team has decided to provide two so called toolchains a year for the most common compiler suits. Each toolchain defines one version of compiler, MPI library, math libraries and other packages as default for all software built aginst this toolchain. On bwUniCluster 3.0 we start with 3 FOSS, 3 Intel and one NVHPC toolchains. For a complete list of EasyBuild toolchains, please see [https://docs.easybuild.io/version-specific/toolchains/ the EasyBuild website]. Currently the 2023a, 2023b, and 2024a toolchains are available, with 2024a being the default one. Some software may not be available for the latest toolchain yet, but may be available for an older toolchain.

	EasyBuild currently supports almost 4000 software packages. We can and will only provide a subset, of course. As a start, we provide current versions of popular software packages ~~form~~ bwUniCluster 2.0. Updating and extending EasyBuild packages is a community effort, which should make providing new software easier. We try to provide additional packages if users request the installation. If we can't provide the software you can still use the advantages of EasyBuild by using it to build your private version.		EasyBuild currently supports almost 4000 software packages. We can and will only provide a subset, of course. As a start, we provide current versions of popular software packages from bwUniCluster 2.0. Updating and extending EasyBuild packages is a community effort, which should make providing new software easier. We try to provide additional packages if users request the installation. If we can't provide the software you can still use the advantages of EasyBuild by using it to build your private version.

	<span id="build-individual-easybuild-packages-as-a-user"></span>		<span id="build-individual-easybuild-packages-as-a-user"></span>
	== Build individual EasyBuild packages as a user ==		== Build individual EasyBuild packages as a user ==

	EasyBuild can also be used to compile your own or niche software, based on the available EasyBuild files by creating an independent EasyBuild software and module tree. The bwUnicluster 3.0 EasyBuild tree can be used as a base to build additional software in your ~~home~~ directory.		EasyBuild can also be used to compile your own or niche software, based on the available EasyBuild files by creating an independent EasyBuild software and module tree. The bwUnicluster 3.0 EasyBuild tree can be used as a base to build additional software in your HOME directory.

	We assume the following		We assume the following

	* the sofware will be installed in your ~~home~~ directory in the sub-directory eb		* the sofware will be installed in your HOME directory in the sub-directory <code>eb</code>
	* you have already cloned the git repository from https://github.com/easybuilders/easybuild-easyconfigs to ~/eb/easybuild-easyconfigs		* you have already cloned the git repository from https://github.com/easybuilders/easybuild-easyconfigs to <code>~/eb/easybuild-easyconfigs</code>
	* create an ~~easybuild~~ configuration file in your home at ~/.config/easybuild/config.cfg		* create an EasyBuild configuration file in your home at <code>~/.config/easybuild/config.cfg</code>

	<pre class="example">[basic]		<pre class="example">[basic]

S Braun: /* Using EasyBuild modules */

2025-03-25T10:12:14Z

Using EasyBuild modules

← Older revision		Revision as of 12:12, 25 March 2025
Line 360:		Line 360:
	== Using EasyBuild modules ==		== Using EasyBuild modules ==

	Users interested in using ~~easyBuild~~ modules need to add the ~~Easybuild~~ module paths to their module environment. This can be done using:		Users interested in using EasyBuild modules need to add the EasyBuild module paths to their module environment. This can be done using:

	<pre class="example">source /opt/bwhpc/common/etc/easybuild/enable_eb_modules		<pre class="example">source /opt/bwhpc/common/etc/easybuild/enable_eb_modules
	</pre>		</pre>
	Now users have acces to a ~~comlete~~ ~~Easybuild~~ module tree, providing compiler toolchains, libraries, and scientific software. The same module commands described above can be used for EasyBuild modules. They can be loaded and unloaded in the same way. Please try not to mix ~~Easybuild~~ modules with the standard modules. There may be unintended consequences. It's always a good idea to [[#purge\|purge]] all modules before loading new ones. If you want to use ~~easybuild~~ modules in your ~~jobscripts~~, please make sure to enable them in your ~~jobscript~~ as well.		Now users have acces to a complete EasyBuild module tree, providing compiler toolchains, libraries, and scientific software. The same module commands described above (Lmod) can be used for EasyBuild modules. They can be loaded and unloaded in the same way. Please try not to mix EasyBuild modules with the standard modules. There may be unintended consequences. It's always a good idea to [[#purge\|purge]] all modules before loading new ones. If you want to use EasyBuild modules in your job scripts, please make sure to enable them in your job script as well.

	To hide the EasyBuild modules again, run:		To hide the EasyBuild modules again, run:
Line 373:		Line 373:

	<span id="easybuild-conventions"></span>		<span id="easybuild-conventions"></span>

	== EasyBuild conventions ==		== EasyBuild conventions ==

S Braun: /* EasyBuild modules */

2025-03-25T10:09:13Z

EasyBuild modules

← Older revision		Revision as of 12:09, 25 March 2025
Line 360:		Line 360:
	== Using EasyBuild modules ==		== Using EasyBuild modules ==

	Users interested in using easyBuild modules need to add the Easybuild module paths to their module environment. This can be done using		Users interested in using easyBuild modules need to add the Easybuild module paths to their module environment. This can be done using:

	<pre class="example">source /opt/bwhpc/common/etc/easybuild/enable_eb_modules		<pre class="example">source /opt/bwhpc/common/etc/easybuild/enable_eb_modules
Line 370:		Line 370:
	<pre class="example">source /opt/bwhpc/common/etc/easybuild/disable_eb_modules		<pre class="example">source /opt/bwhpc/common/etc/easybuild/disable_eb_modules
	</pre>		</pre>
	or start again with a clean shell		or start again with a clean shell.

	<span id="easybuild-conventions"></span>		<span id="easybuild-conventions"></span>

P Schuhmacher: /* EasyBuild modules */

2025-02-21T14:27:25Z

EasyBuild modules

← Older revision		Revision as of 16:27, 21 February 2025
Line 353:		Line 353:
	</pre>		</pre>
	<span id="EasyBuild"></span>		<span id="EasyBuild"></span>
	= ~~<code>~~EasyBuild modules~~</code>~~ =		= EasyBuild modules =

	With bwUniCluster 3.0 we start an official beta-test to provide toolchains and scientific software built with [https://docs.easybuild.io/ EasyBuild]. EasyBuild is a well established and consistent way to provide scientific software at many HPC sites all over the world. Since the naming of Easybuild modules is consistent users can use modules with the same name at different sites. To make this possible, Easybuild software minimizes the resources used by the local operating system and provides most software independently. This makes scientific results more reproducible.		With bwUniCluster 3.0 we start an official beta-test to provide toolchains and scientific software built with [https://docs.easybuild.io/ EasyBuild]. EasyBuild is a well established and consistent way to provide scientific software at many HPC sites all over the world. Since the naming of Easybuild modules is consistent users can use modules with the same name at different sites. To make this possible, Easybuild software minimizes the resources used by the local operating system and provides most software independently. This makes scientific results more reproducible.
Line 431:		Line 431:
	</pre>		</pre>
	<span id="how_do_modules_work"></span>		<span id="how_do_modules_work"></span>

	= How do Modules work? =		= How do Modules work? =

P Schuhmacher at 14:27, 21 February 2025

2025-02-21T14:27:10Z

← Older revision		Revision as of 16:27, 21 February 2025
Line 4:		Line 4:
	'''Software (Environment) Modules''', or short '''modules''' are the means by which most of the installed scientific software is provided on ''bwUniCluster 3.0''.		'''Software (Environment) Modules''', or short '''modules''' are the means by which most of the installed scientific software is provided on ''bwUniCluster 3.0''.

	~~The use of different~~ compilers, libraries and software packages requires users to set up a specific session environment suited for the program they want to run. ''bwUniCluster 3.0'' provides users with the possibility to load and unload complete environments for compilers, libraries and software packages using a single command.		Different compilers, libraries and software packages requires users to set up a specific session environment suited for the program they want to run. ''bwUniCluster 3.0'' provides users with the possibility to load and unload complete environments for compilers, libraries and software packages using a single command.

	<span id="basics"></span>		<span id="basics"></span>
	= Basics =		= Basics =

	The Environment ''modules'' package enables dynamic modification of ~~your~~ ~~environment~~. Typically, a modulefile contains instructions that alter or set shell environment variables, such as PATH and MANPATH, to enable access to various installed software.		The Environment ''modules'' package enables dynamic modification of shell environments. Typically, a modulefile contains instructions that alter or set shell environment variables, such as PATH and MANPATH, to enable access to various installed software.

	One of the key features of ~~using~~ Environment ''Modules'' is to ~~allow~~ multiple versions of the same software ~~to be used in your environment~~ in a controlled manner. For example, two different versions of the Intel C compiler can be installed on the system at the same time - the version used is based upon which Intel C compiler modulefile is loaded.		One of the key features of Environment ''Modules'' is the ability to have multiple versions of the same software available in a controlled manner. For example, two different versions of the Intel C compiler can be installed on the system at the same time - the version used is based upon which Intel C compiler modulefile is loaded.

	The software stack of bwUniCluster 3.0 provides a number of modules. Users can also ~~create~~ their own ~~''modules''~~. ''Modules'' may ~~even~~ be shared by users on a system, and users may have their own collection of ''modules'' to supplement or replace the global ''modules''.		The software stack of bwUniCluster 3.0 provides a number of modules. Users can also provide modules for their own software. ''Modules'' may be shared between users on a system, and users may have their own collection of ''modules'' to supplement or replace the global ''modules''.

	To see which modules are currently loaded/enabled one must execute		To see which modules are currently loaded/enabled one must execute
Line 33:		Line 33:
	= Usage =		= Usage =

	bwUniCluster 3.0 has been set up to use the ''Lmod'' module system. ''Lmod'' is a modern replacement of the older environment modules. The complete ''Lmod'' documentation can be found [[<nowiki/>http://lmod.readthedocs.org/en/latest/][here]].		bwUniCluster 3.0 has been set up to use the ''Lmod'' module system. ''Lmod'' is a modern replacement of the older tcl-based environment modules. The complete ''Lmod'' documentation can be found [[<nowiki/>http://lmod.readthedocs.org/en/latest/][here]].

	<span id="documentation"></span>		<span id="documentation"></span>
Line 46:		Line 46:
	-h -? -H --help This help message		-h -? -H --help This help message
	...		...
	a lot of ~~output~~		a lot of help
	...		...
	Lmod Web Sites		Lmod Web Sites
Line 101:		Line 101:
	== Display all available Modules ==		== Display all available Modules ==

	bwUniCluster3.0 comes with a variety of modules pre-installed. Some of these modules may not be available for all users, due to licensing restrictions. You can display all available ''modules'' by running:		bwUniCluster3.0 comes with a variety of modules pre-installed. Some of these modules may not be available for all users of all institutions, due to licensing restrictions. You can display all available ''modules'' by running:

	<pre class="example">$ module avail		<pre class="example">$ module avail
Line 125:		Line 125:
	== Finding software Modules ==		== Finding software Modules ==

	Currently all bwUniCluster 3.0 software packages are assigned to the following ''Module'' categories ~~(???)~~:		Currently all bwUniCluster 3.0 software packages are assigned to the following ''Module'' categories:

	* bio		* bio
Line 353:		Line 353:
	</pre>		</pre>
	<span id="EasyBuild"></span>		<span id="EasyBuild"></span>
	= EasyBuild modules =		= <code>EasyBuild modules</code> =

	With bwUniCluster 3.0 we start an official beta-test to provide toolchains and scientific software built with [https://docs.easybuild.io/ EasyBuild]. EasyBuild is a well established and consistent way to provide scientific software at many HPC sites all over the world. Since the naming of Easybuild modules is consistent users can use modules with the same name at different sites. To make this possible, Easybuild software minimizes the resources used by the local operating system and provides most software independently. This makes scientific results more reproducible.		With bwUniCluster 3.0 we start an official beta-test to provide toolchains and scientific software built with [https://docs.easybuild.io/ EasyBuild]. EasyBuild is a well established and consistent way to provide scientific software at many HPC sites all over the world. Since the naming of Easybuild modules is consistent users can use modules with the same name at different sites. To make this possible, Easybuild software minimizes the resources used by the local operating system and provides most software independently. This makes scientific results more reproducible.
Line 388:		Line 388:
	* the sofware will be installed in your home directory in the sub-directory eb		* the sofware will be installed in your home directory in the sub-directory eb
	* you have already cloned the git repository from https://github.com/easybuilders/easybuild-easyconfigs to ~/eb/easybuild-easyconfigs		* you have already cloned the git repository from https://github.com/easybuilders/easybuild-easyconfigs to ~/eb/easybuild-easyconfigs
	* create an easybuild configuration file		* create an easybuild configuration file in your home at ~/.config/easybuild/config.cfg

	<pre class="example">[basic]		<pre class="example">[basic]
Line 431:		Line 431:
	</pre>		</pre>
	<span id="how_do_modules_work"></span>		<span id="how_do_modules_work"></span>

	= How do Modules work? =		= How do Modules work? =