bwHPC Wiki - Recent changes [en]

FAQ

K Siegmund — Mon, 22 Jun 2026 13:31:20 GMT

Access and Account Issues: bwForCluster

← Older revision		Revision as of 15:31, 22 June 2026
(One intermediate revision by the same user not shown)
Line 40:		Line 40:
	Please check which of those two things are missing using the following documentation:		Please check which of those two things are missing using the following documentation:

	==== Entitlement ====		==== Entitlement (and IDP Problems) ====

	You need the bwForCluster Entitlement.		You need the bwForCluster Entitlement.

			The same page can also help you recognize problems with the IDP (identity provider) of your home university

	<b>→</b> Read [[Registration/bwForCluster/Entitlement]] and check if it is present.		<b>→</b> Read [[Registration/bwForCluster/Entitlement]] and check if it is present.

			In case
⚫	~~If the entitlement is missing:~~ please check with the local support of your home university (the university where you study or are employed)
			* the entitlement is missing
			* Name/Email/eduPersonPrincipalName in the "personal data" tab are incorrect or missing
		⚫	<b>→</b> please check with the local support of your home university (the university where you study or are employed)

			Note: in case of temporary problems with your IDP, data in the registration server can become corrupted or missing. Logging in to the registration server that your cluster uses can fix this problem and fill in correct values again. Check in [[Registration/Password]] which registration server your cluster uses and log in there (for some cluster it is the same as given for the Entitlement Check).

	==== ZAS Permission ====		==== ZAS Permission ====

Registration/bwForCluster/Entitlement

K Siegmund — Mon, 22 Jun 2026 13:22:01 GMT

Check your Entitlements

← Older revision		Revision as of 15:22, 22 June 2026
Line 18:		Line 18:
	== Check your Entitlements ==		== Check your Entitlements ==

⚫	First check if you already have the Entitlement, some universities automatically assign the entitlement to some accounts:

		⚫	First check if you already have the Entitlement, some universities automatically assign the entitlement to some accounts (you must allow that your data is being transferred to that page or your cluster login will also not work):
⚫	→ '''[https://login.bwidm.de/user/index.xhtml?show=entitlement&highlight=bwForCluster Registration Server bwForCluster Entitlement Check]'''

		⚫	→ '''[https://login.bwidm.de/user/index.xhtml?show=entitlement&highlight=bwForCluster Registration Server bwForCluster Entitlement Check]'''
	<!-- image can be deleted [[File:BwIDM-idp.png\|center\|600px\|thumb\|Verify Entitlement.]] -->

	Note: if you cannot log in to the registration server at all, some crucial information is missing in the information your home university is transmitting to it. Contact the helpdesk of your home university.		Note: if you cannot log in to the registration server at all, some crucial information is missing in the information your home university is transmitting to it. Contact the helpdesk of your home university.

File:BwIDM-idp.png

K Siegmund — Mon, 22 Jun 2026 13:20:35 GMT

K Siegmund deleted page File:BwIDM-idp.png replaced by tab showing only entitlement

Registration/bwForCluster/RV/Management

H Schumacher — Mon, 22 Jun 2026 10:59:31 GMT

added RV extension section, redesigned 'change RVV' section

@@ Line 44: / Line 44: @@
 ==== Change the RV Responsible ====
-Prerequisites:
+<ins>Prerequisites:</ins>
 * You need to be the current RV responsible. In case you don't have access to your account anymore, you can ask the cluster support for help.
-* The person you want to appoint new responsible is a member of the RV.
+* The person you want to appoint as new RV responsible is a member of the RV.
-Steps:
+<ins>Steps:</ins>
-[[File:RV-my.png|center|frame|My RVs.]]
+File:RV-my.png|center|frame|'''1.''' Login to [https://zas.bwhpc.de/shib/en/info_rv.php '''My RVs'''] and go to the RV.

DACHS/Jupyter

M Kunzelmann — Fri, 19 Jun 2026 11:59:40 GMT

Prerequisites

← Older revision		Revision as of 13:59, 19 June 2026
Line 17:		Line 17:
	== Prerequisites ==		== Prerequisites ==
	# Register as described on the [[DACHS/Registration\|Registration]]		# Register as described on the [[DACHS/Registration\|Registration]]
	# Then [[DACHS/Login\|login to DACHS]] via SSH '''at least once''': This ensures that your home directory is setup properly on DACHS
	# Make sure you're '''connected to the VPN of your university''' (see above)		# Make sure you're '''connected to the VPN of your university''' (see above)


	= Login Process =		= Login Process =

Justus3/Getting Started

K Siegmund — Fri, 19 Jun 2026 11:21:17 GMT

Created page with " == Pre-Requisite: Get Cluster Access == Follow the registration process for the bwForCluster. → How to Register for a bwForCluster == Pre-Requisite: Know the Workflow of Running Calculations == On a compute cluster, you do not simply run log in and your software, but you write a "job script" that contains all commands to run and process your job and send this into a waiting queue to be run on one of several hundred computers. Ho..."

New page

== Pre-Requisite: Get Cluster Access ==

Follow the registration process for the bwForCluster.

→ [[Registration/bwForCluster|How to Register for a bwForCluster]]

== Pre-Requisite: Know the Workflow of Running Calculations ==

On a compute cluster, you do not simply run log in and your software, but you write a "job script" that contains all commands to run and process your job and send this into a waiting queue to be run on one of several hundred computers.

How this is done is described in a little more detail here: [[Running Calculations]]

== Login to the Cluster ==

Setup service password and 2FA token and login to the cluster. → [[JUSTUS2/Login|Login JUSTUS2]]

== Online Course on Justus2 ==
* Training course → [https://training.bwhpc.de/goto.php?target=crs_629_rcodeM6n48kAUsT&client_id=BWHPC Introduction to JUSTUS2 ]

== Know the Cluster Rules ==

→ '''[[JUSTUS2/Policy_Agreement]]'''

== Using the Linux Commandline ==

* [[Media:HPC_User_Linux_command_cheatsheet.pdf|HPC User's Linux Cheat Sheet (PDF)]]

* HPC Wiki (external site) → [https://hpc-wiki.info/hpc/Introduction_to_Linux_in_HPC/The_Command_Line Introduction to Linux Commandline]

* Training course → [https://training.bwhpc.de/ilias.php?baseClass=illmpresentationgui&cmd=resume&ref_id=310 Linux course on training.bwhpc.de] (please log in to the training site, then click this link again)

also see: [[.bashrc Do's and Don'ts]]

== Transfer your Data to the Cluster ==

Get familiar with available file systems on the cluster. → [[Hardware_and_Architecture_(bwForCluster_JUSTUS_2)#Storage_Architecture|File Systems]]

Transfer your data to the cluster using appropriate tools. → [[Data Transfer|Data Transfer]]

== Find Information About Installed Software and Examples ==

Compiler, Libraries and application software are provided as software modules. Learn how to work with
[[Environment_Modules|software modules]]. → [[JUSTUS2/Software|Software]]


Run sample script from a pre-installed software (Software job examples in the page above)

== Run your Software in a Batch Job ==

Get familiar with available nodes types on the cluster. → [[Hardware_and_Architecture_(bwForCluster_JUSTUS_2)|Hardware and Architecture]]

Submit and monitor your jobs with Slurm commands.
* → [[JUSTUS2/Jobscripts: Running Your Calculations|Jobscripts: Running Your Calculations]]
* → [[BwForCluster_JUSTUS_2_Slurm_HOWTO| extensive Slurm HOWTO on specific tasks]]

== Learn about Scaling your Job ==

How many compute-cores should my job use? This depends on the software and the problem you are trying to solve. But if you use too few cores, your computation may take much too long - if you use too many cores, they will not improve the speed of your computation and all you do by using more cores is wasting compute resources and energy.

If you run hundreds or thousands of similar calculations, you should look at this carefully before starting.

How to do this is described in: [[Scaling]]

== Acknowledge the Cluster ==

Remember to mention the cluster in your publications. → [[bwForCluster JUSTUS 2 Acknowledgement|Acknowledgement]]

Justus3

K Siegmund — Fri, 19 Jun 2026 09:15:33 GMT

← Older revision		Revision as of 11:15, 19 June 2026
Line 3:		Line 3:

	The '''bwForCluster Justus 3''' is the state-wide high-performance compute cluster dedicated to '''Computational Chemistry and Quantum Sciences''' in Baden-Württemberg.		The '''bwForCluster Justus 3''' is the state-wide high-performance compute cluster dedicated to '''Computational Chemistry and Quantum Sciences''' in Baden-Württemberg.

	<!--{\| style=" background:#FEF4AB; width:100%;"
	\| style="padding:8px; background:#FFE856; font-size:120%; font-weight:bold; text-align:left" \| News
	\|-
	\|
	\|}
	-->
	{\| style=" background:#FEF4AB; width:100%;"		{\| style=" background:#FEF4AB; width:100%;"
	\| style="padding:8px; background:#FFE856; font-size:120%; font-weight:bold; text-align:left" \| News		\| style="padding:8px; background:#FFE856; font-size:120%; font-weight:bold; text-align:left" \| News
	\|-		\|-
	\|		\|
			* 2026-06-16: Cooling installations for the new cluster has been enabled
	* None
	\|}		\|}
	{\| style=" background:#eeeefe; width:100%;"		{\| style=" background:#eeeefe; width:100%;"

Justus3

K Siegmund — Fri, 19 Jun 2026 09:06:28 GMT

Created page with " 700px The '''bwForCluster Justus 3''' is the state-wide high-performance compute cluster dedicated to '''Computational Chemistry and Quantum Sciences''' in Baden-Württemberg.  {| style=" b..."

New page

[[File:Justus3_header.png|700px]]

The '''bwForCluster Justus 3''' is the state-wide high-performance compute cluster dedicated to '''Computational Chemistry and Quantum Sciences''' in Baden-Württemberg.

{| style=" background:#FEF4AB; width:100%;"
| style="padding:8px; background:#FFE856; font-size:120%; font-weight:bold; text-align:left" | News
|-
|
* None
|}
{| style=" background:#eeeefe; width:100%;"
| style="padding:8px; background:#dedefe; font-size:120%; font-weight:bold; text-align:left" | Training & Support
|-
|
* [[Justus3/Getting Started|Getting Started]]
* Online Tutorial [https://hpc-wiki.info/hpc/Introduction_to_Linux_in_HPC Introduction to Linux in HPC] (external link)
* [[Justus3/Support|Contact and Support]]
* Send [[Feedback|Feedback]] about Wiki pages
|}

{| style=" background:#deffee; width:100%;"
| style="padding:8px; background:#cef2e0; font-size:120%; font-weight:bold; text-align:left" | User Documentation
|-
|
* [[Registration/bwForCluster|Registration]]
* [[Justus3/Policy Agreement|Policy Agreement]]

* [[Justus3/Login|Login]]
* [[Justus3/Hardware|Hardware and Architecture]]
** [[Justus3/Hardware#Node_Specifications|Node Specifications]]
** [[Justus3/Hardware#Storage_Architecture|File Systems and Workspaces]]

* [[Justus3/Software|Software]]
* [[Justus3/Jobscripts: Running Your Calculations|Jobscripts: Running Your Calculations]]
** [[bwForCluster Justus 3 Slurm HOWTO|Slurm HOWTO]]
* [[Justus3/Visualization|Visualisation]]
* [[Development]]
|}

{| style=" background:#e6e9eb; width:100%;"
| style="padding:8px; background:#d1dadf; font-size:120%; font-weight:bold; text-align:left" | Cluster Funding
|-
|
* Please '''[[Justus3/Acknowledgement|acknowledge]]''' the Justus 3 cluster in your publications.
|}

Status

K Siegmund — Fri, 19 Jun 2026 08:30:23 GMT

← Older revision		Revision as of 10:30, 19 June 2026
Line 6:		Line 6:
	Due to work on the power supply of the cooling system all jobs have been suspended to prevent overheating.-->		Due to work on the power supply of the cooling system all jobs have been suspended to prevent overheating.-->

	{{Template:Status\|~~warn~~\|Status: ~~BinAC 2 maintenance: June 8–12~~ }}		{{Template:Status\|ok\|Status: Ok}}

	<!--		<!--
Line 16:		Line 16:

	== Relevant Old Messages ==		== Relevant Old Messages ==
			* BinAC 2 maintenance: June 8–12
	None

Main Page

K Siegmund — Fri, 19 Jun 2026 08:29:53 GMT

← Older revision		Revision as of 10:29, 19 June 2026
Line 10:		Line 10:
	<!-- bwHPC STATUS START -->		<!-- bwHPC STATUS START -->

	~~<!--~~{{Template:Status\|~~warn~~\|Status: ~~Jobs suspended on Justus 2~~}}~~-->~~		{{Template:Status\|ok\|Status: Ok}}
⚫	{{Template:Status\|warn\|Status: ~~BinAC~~ 2 ~~maintenance:~~ ~~June 8–12~~ }}

	<!--		<!--
Line 18:		Line 17:
	{{Template:Status\|warn\|Status: maintenance upcoming}}		{{Template:Status\|warn\|Status: maintenance upcoming}}
	{{Template:Status\|alert\|Status: bwUniCluster down}}		{{Template:Status\|alert\|Status: bwUniCluster down}}
		⚫	{{Template:Status\|warn\|Status: Jobs suspended on Justus 2}}
	-->		-->

Data Transfer/SSHFS

K Siegmund — Wed, 17 Jun 2026 14:26:17 GMT

@@ Line 15: / Line 15: @@
 == Usage ==
 === Windows ===
@@ Line 34: / Line 60: @@
 # subfolderX
 sshfs <username>@<hostname>:subfolderX <localPath></syntaxhighlight></li></ol>

Registration/bwForCluster/Helix

H Schumacher — Wed, 17 Jun 2026 08:43:27 GMT

emphasized main point

← Older revision		Revision as of 10:43, 17 June 2026
Line 43:		Line 43:
	=== Troubleshooting with the Help of bwServices ===		=== Troubleshooting with the Help of bwServices ===
	When encountering connection issues with a service, the bwServices website is a good starting point for troubleshooting:		When encountering connection issues with a service, the bwServices website is a good starting point for troubleshooting:
			{\| style="border: 2px solid #FFD28A; background-color: #ffe0af; padding: 10px; margin-bottom: 1em; width: 100%;"
			\|-
		⚫	\| '''⚠ Login to bwServices first.''' This triggers an update of your user information within some minutes and solves the connection issue in the following cases:
			* User data changed: entitlement, SV/RV status, SV/RV membership status, role or email address changed or uni-id was (re)activated
			* Synchronization issues of the service
			\|}

	* <ins>'''Issue:</ins> Expired contract''' or inactive student id:		* <ins>'''Issue:</ins> Expired contract''' or inactive student id:
	** No access to state services like the bwHPC clusters or SDS@hd is possible.		** No access to state services like the bwHPC clusters or SDS@hd is possible.
	** You can see that this is the issue when the login to bwServices via your identity provider is not possible.		** You can see that this is the issue when the login to bwServices via your identity provider is not possible.
	* <ins>'''Issue:</ins> Recently (re)activated uni-id or other user data changes:''' Your id at your home organization was (re)activated a short time ago or your email address changed (or sth. similar) but this change didn't get through to the bwHPC cluster or SDS@hd service.
⚫	** Login to bwServices. This triggers an update of your user information. ~~Some~~ minutes ~~later,~~ the connection to the ~~service~~ ~~should work again.~~
	* <ins>'''Issue:</ins> Entitlement / RV missing:''' Prerequisites like entitlement ownership or RV membership aren't met (anymore).		* <ins>'''Issue:</ins> Entitlement / RV missing:''' Prerequisites like entitlement ownership or RV membership aren't met (anymore).
	** You can see the concrete issue and advice on how to move forward when looking at the registry info of the service on the bwServices website.		** You can see the concrete issue and advice on how to move forward when looking at the registry info of the service on the bwServices website.

NEMO2/Easybuild Modules

M Janczyk — Tue, 16 Jun 2026 15:16:17 GMT

Physical vs. Symbolic Architectures

← Older revision		Revision as of 17:16, 16 June 2026
(2 intermediate revisions by the same user not shown)
Line 30:		Line 30:
	* <code>arch/l40s</code> - NVIDIA L40S GPUs (Intel-based nodes)		* <code>arch/l40s</code> - NVIDIA L40S GPUs (Intel-based nodes)
	* <code>arch/h200</code> - NVIDIA H200 GPUs		* <code>arch/h200</code> - NVIDIA H200 GPUs
			* <code>arch/rtx</code> - NVIDIA RTX PRO 6000 Blackwell GPUs

	== Physical vs. Symbolic Architectures ==		== Physical vs. Symbolic Architectures ==
Line 48:		Line 49:
	\|-		\|-
	\| <code>h200</code> \|\| AMD Genoa CPUs + NVIDIA H200 GPUs \|\| '''Symbolic link → genoa''' \|\| Uses genoa installation directory		\| <code>h200</code> \|\| AMD Genoa CPUs + NVIDIA H200 GPUs \|\| '''Symbolic link → genoa''' \|\| Uses genoa installation directory
			\|-
			\| <code>rtx</code> \|\| AMD Genoa CPUs + NVIDIA RTX PRO 6000 Blackwell GPUs \|\| '''Symbolic link → genoa''' \|\| Uses genoa installation directory
	\|}		\|}

	'''What this means for system modules (<code>/opt/eb</code>):''' The global NEMO2 modules use symbolic links, so modules for <code>mi300a</code> and <code>~~h200~~</code> are actually stored in the <code>genoa</code> directory. This is because these GPU nodes use AMD Genoa CPUs as their host processors.		'''What this means for system modules (<code>/opt/eb</code>):''' The global NEMO2 modules use symbolic links, so modules for <code>mi300a</code>, <code>h200</code> and <code>rtx</code> are actually stored in the <code>genoa</code> directory. This is because these GPU nodes use AMD Genoa CPUs as their host processors.

	'''What this means for self-built modules:''' When you build your own modules with <code>eb-build-module.sh</code>, each architecture gets its own separate directory by default:		'''What this means for self-built modules:''' When you build your own modules with <code>eb-build-module.sh</code>, each architecture gets its own separate directory by default:
Line 58:		Line 61:
	* <code>~/.local/easybuild/l40s/</code>		* <code>~/.local/easybuild/l40s/</code>
	* <code>~/.local/easybuild/h200/</code>		* <code>~/.local/easybuild/h200/</code>
			* <code>~/.local/easybuild/rtx/</code>

	'''Optional:''' You can manually create symbolic links (<code>mi300a → genoa</code>, <code>h200 → genoa</code>) to mirror the global structure, but this is not required. See [[NEMO2/Easybuild_Modules/EB_Build_Module#Architecture_Support\|Architecture Support]] for details.		'''Optional:''' You can manually create symbolic links (<code>mi300a → genoa</code>, <code>h200 → genoa</code>, <code>rtx → genoa</code>) to mirror the global structure, but this is not required. See [[NEMO2/Easybuild_Modules/EB_Build_Module#Architecture_Support\|Architecture Support]] for details.

	== Viewing Modules for Specific Architectures ==		== Viewing Modules for Specific Architectures ==

NEMO2/Hardware

M Janczyk — Tue, 16 Jun 2026 15:09:53 GMT

Operating System and Software

← Older revision		Revision as of 17:09, 16 June 2026
(One intermediate revision by the same user not shown)
Line 13:		Line 13:
	\|-		\|-
	!scope="column" \| Own Software Modules using EasyBuild and Spack		!scope="column" \| Own Software Modules using EasyBuild and Spack
			\| [[NEMO2/Easybuild_Modules\| Easybuild]]
	\| [https://docs.easybuild.io/ EasyBuild]
	\|-		\|-
	!scope="column" \| Own (Python) Environments with Conda		!scope="column" \| Own (Python) Environments with Conda
Line 25:		Line 25:

	__TOC__		__TOC__


	== Compute and Special Purpose Nodes ==		== Compute and Special Purpose Nodes ==

NEMO2/Slurm

M Janczyk — Tue, 16 Jun 2026 15:04:49 GMT

Accounting

@@ Line 186: / Line 186: @@
 == Show free nodes ==
-The Slurm command <code>sinfo</code> is used to view partition and node information for a system running Slurm, unfortunately it can only be used by the administrators. For users there is a special binary <code>sinfo_t_idle</code> to find out how many nodes are available on the system for immediate use.
+The Slurm command <code>sinfo</code> is used to view partition and node information for a system running Slurm.
 <pre>
-Partition genoa                   :     30 nodes idle
+cpu*         up 4-00:00:00      2  idle  n[3303-3304]
-Partition milan                   :     97 nodes idle
+genoa        up 4-00:00:00      0    n/a
-Partition l40s                    :      6 nodes idle
+milan        up 4-00:00:00      2  idle  n[3303-3304]
-Partition mi300a                  :      3 nodes idle
+l40s         up 2-00:00:00      0    n/a
-Partition login                   :      2 nodes idle
+h200         up 2-00:00:00      0    n/a
 </pre>
@@ Line 448: / Line 451: @@
 = Accounting =

Energy Efficient Cluster Usage

K Siegmund — Tue, 16 Jun 2026 10:07:25 GMT

How many and which kind of hardware resources do I require for it

← Older revision		Revision as of 12:07, 16 June 2026
Line 34:		Line 34:
	As this is dependent on both the job as well as the specific cluster hardware and architecture available.		As this is dependent on both the job as well as the specific cluster hardware and architecture available.

		⚫	'''A small number of jobs and few resources'''
	'''
⚫	A small number of jobs and few resources'''

	→ Submit to the scheduler. No extended testing and resource scaling analysis are needed.		→ Submit to the scheduler. No extended testing and resource scaling analysis are needed.

		⚫	'''Medium-sized projects'''
	'''
⚫	Medium-sized projects'''

	→ Run only necessary jobs: Please consider testing new setups and their output for validity prior to submitting a huge amount of similar jobs		→ Run only necessary jobs: Please consider testing new setups and their output for validity prior to submitting a huge amount of similar jobs

NEMO2/Slurm

M Janczyk — Tue, 16 Jun 2026 09:53:00 GMT

Partitions

← Older revision		Revision as of 11:53, 16 June 2026
(2 intermediate revisions by the same user not shown)
Line 84:		Line 84:
	'''GPUs'''		'''GPUs'''

	GPUs are requested as "generic resources" with <code>--gres:gpu:<count></code> or <code>--gpus:<count></code>.		GPUs are requested as "generic resources" with <code>--gres=gpu:<count></code> or <code>--gpus=<count></code>.

	== Default Values ==		== Default Values ==
Line 104:		Line 104:
	'''All''' partitions are operated in shared mode, i.e. jobs from different users can run on the same node. Jobs can get exclusive access to compute nodes in these partitions with the <code>--exclusive</code> option.		'''All''' partitions are operated in shared mode, i.e. jobs from different users can run on the same node. Jobs can get exclusive access to compute nodes in these partitions with the <code>--exclusive</code> option.

	The login partition is intended for quick tests on the login ~~lodes~~ with the Slurm environment.		The login partition is intended for quick tests on the login nodes with the Slurm environment.

	See [[NEMO2/Hardware]] for more information on the exact hardware specifications.		See [[NEMO2/Hardware]] for more information on the exact hardware specifications.
Line 175:		Line 175:
	\|-		\|-
	\| login		\| login
	\| ~~single~~ node		\| multi node
	\| 1x AMD EPYC 9354 (Genoa) (SMT on)		\| 1x AMD EPYC 9354 (Genoa) (SMT on)
	\| ---		\| ---
	\| ---		\| ---
	\| ntasks=1, time=15:00, mem-per-cpu=1gb		\| ntasks=1, time=15:00, mem-per-cpu=1gb
	\| time=15:00, nodes=1, ntasks=4, mem=50gb		\| time=15:00, nodes=2, ntasks=4, mem=50gb
	\|}		\|}
	<code>*</code> default partition		<code>*</code> default partition
Line 264:		Line 264:

	'''Notes:'''		'''Notes:'''
	* Jobs with <code>--mem</code> up to ~~500gb~~ can run on all node types associated with the cpu partition.		* Jobs with <code>--mem</code> up to 495GB can run on all node types associated with the cpu partition (routing to either milan or genoa nodes).

	=== Multi-threaded Programs ===		=== Multi-threaded Programs ===
Line 318:		Line 318:
	== Interactive Tests on the Login Nodes ==		== Interactive Tests on the Login Nodes ==

	The login partition is intended for quick tests on the login ~~lodes~~ with the Slurm environment. The limits are 4 cores and 15 minutes of walltime.		The login partition is intended for quick tests on the login nodes with the Slurm environment. The limits are 4 cores and 15 minutes of walltime.

	<syntaxhighlight lang="shell">		<syntaxhighlight lang="shell">
Line 402:		Line 402:
	= Queue / Partition Limits =		= Queue / Partition Limits =

			'''Per-user CPU and core-minute limits depend on the partition/QOS:'''
	'''Currently, each user can use 8000 cores (QOSMaxCpuPerUserLimit) and 7,2 million core minutes (MaxCpuRunMinsPerUser) at any time.'''

			{\| class="wikitable"
⚫	The core minutes ~~can~~ be calculated by (running CPU cores) * (remaining runtime of all jobs in minutes) = ~~CpuRunMinsPerUser,~~ ~~example~~:
			\|-
	<code>5000cores * 24h * 60min = 7.200.000 CpuRunMinsPerUser</code>
			! Partition/QOS !! Max Cores (QOSMaxCpuPerUserLimit) !! Max Core-Minutes Running (MaxCpuRunMinsPerUser)
			\|-
			\| cpu (default) \|\| 12,000 \|\| 14,500,800
			\|-
			\| genoa \|\| 8,000 \|\| 14,500,800
			\|-
			\| milan \|\| 8,000 \|\| 12,428,640
			\|}

		⚫	The core-minutes limit is calculated as (running CPU cores) × (remaining runtime of all jobs in minutes). Example for genoa:
			<code>8000 cores × 30h × 60min = 14,400,000 core-minutes</code>

	This is constantly evaluated, you do not have to do anything.		This is constantly evaluated, you do not have to do anything.
	This value decreases after a few minutes (running CPU cores * remaining runtime of all jobs in minutes) or when a job ends.		This value decreases after a few minutes (running CPU cores * remaining runtime of all jobs in minutes) or when a job ends.

	'''The partitions 'cpu', 'l40s', 'mi300a' and '~~h200~~' only allow the use of single nodes.'''		'''The partitions 'cpu', 'l40s', 'mi300a', 'h200' and 'rtx' only allow the use of single nodes.'''
	You cannot start jobs that use more than one node, either by specifying <code>--nodes</code> (>1) or <code>--ntasks</code> (more than available on one node).		You cannot start jobs that use more than one node, either by specifying <code>--nodes</code> (>1) or <code>--ntasks</code> (more than available on one node).
	If you do so, your jobs will be blocked with the reason (PartitionNodeLimit).		If you do so, your jobs will be blocked with the reason (PartitionNodeLimit).
Line 428:		Line 439:
	\| QOSMaxCpuPerUserLimit		\| QOSMaxCpuPerUserLimit
	\| Your job cannot be started as your user would otherwise exceed the maximum permitted cores for one user.		\| Your job cannot be started as your user would otherwise exceed the maximum permitted cores for one user.
			\|-
			\| QOSMaxGRESPerUser
			\| Your job exceeds the per-user GPU limit (l40s: 16 GPUs, h200: 8 GPUs, rtx: 4 GPUs, mi300a: 8 GPUs).
	\|}		\|}

	Check additional [https://slurm.schedmd.com/job_reason_codes.html reason codes] at Schedmd.		Check additional [https://slurm.schedmd.com/job_reason_codes.html reason codes] at Schedmd.

	= Accounting ~~(not implemented)~~ =		= Accounting =

	Jobs are billed for allocated CPU cores, memory and GPUs.		Jobs are billed for allocated CPU cores, memory and GPUs.

NEMO2/Hardware

M Janczyk — Tue, 16 Jun 2026 09:27:14 GMT

← Older revision		Revision as of 11:27, 16 June 2026
Line 18:		Line 18:
	\| [[Conda]]		\| [[Conda]]
	\|-		\|-
	!scope="column" \| Containers with ~~Apptianer~~/Singularity (and enroot in the future)		!scope="column" \| Containers with Apptainer/Singularity (and enroot in the future)
	\| [[Development/Containers]]		\| [[Development/Containers]]
	\|-		\|-
Line 29:		Line 29:
	== Compute and Special Purpose Nodes ==		== Compute and Special Purpose Nodes ==

	For researchers from the scientific fields '''N'''euroscience, '''E'''lementary Particle Physics, '''M'''icrosystems Engineering and '''M'''aterials Science the bwForCluster '''NEMO''' offers about ~~240~~ compute nodes plus several special purpose nodes for login, interactive jobs, visualization, machine learning and ai.		For researchers from the scientific fields '''N'''euroscience, '''E'''lementary Particle Physics, '''M'''icrosystems Engineering and '''M'''aterials Science the bwForCluster '''NEMO''' offers about 260 compute nodes plus several special purpose nodes for login, interactive jobs, visualization, machine learning and ai.

	Node specification (see [[NEMO2/Slurm]] for slurm partitons):		Node specification (see [[NEMO2/Slurm]] for slurm partitons):
Line 154:		Line 154:
	!scope="column" \| Job Example MI300A Partition		!scope="column" \| Job Example MI300A Partition
	\|colspan="7" \| Maximum resources for a single node job (*):		\|colspan="7" \| Maximum resources for a single node job (*):
	<code>--partition=mi300a --ntasks=94 --gres=gpu:4 --mem=495GB # or: --mem=507000MB, or: --mem-per-cpu=5300MB</code>		<code>--partition=mi300a --ntasks=92 --gres=gpu:4 --mem=495GB # or: --mem=507000MB, or: --mem-per-cpu=5300MB</code>
	\|-		\|-
	!scope="column" \| Job Example H200 Partition		!scope="column" \| Job Example H200 Partition
Line 162:		Line 162:
	!scope="column" \| Job Example RTX Partition		!scope="column" \| Job Example RTX Partition
	\|colspan="7" \| Maximum resources for a single node job (*):		\|colspan="7" \| Maximum resources for a single node job (*):
	<code>--partition=rtx --ntasks=190 --gres=gpu:4 --mem=~~727GB~~ # or: --mem=~~745000MB~~, or: --mem-per-cpu=~~3900MB~~</code>		<code>--partition=rtx --ntasks=190 --gres=gpu:4 --mem=1459GB # or: --mem=1495000MB, or: --mem-per-cpu=7800MB</code>
	\|-		\|-
	!scope="column" \| Job Example Login Partition		!scope="column" \| Job Example Login Partition
Line 175:		Line 175:
	NEMO2 offers a fast [https://docs.weka.io/ Weka] parallel filesystem, which is only limited by the uplink to this storage (>90GB/s).		NEMO2 offers a fast [https://docs.weka.io/ Weka] parallel filesystem, which is only limited by the uplink to this storage (>90GB/s).
	The storage is used for <tt>$HOME</tt> and '''[[workspace]]s'''.		The storage is used for <tt>$HOME</tt> and '''[[workspace]]s'''.
	There ~~sill~~ be no backups~~, but we plan to implement~~ Snapshots for ~~the last 7 days~~ in ~~the~~ ~~next~~ ~~months~~.		There will be no backups. Snapshots for <tt>$HOME</tt> are available (see below).
	Additionally, each compute node provides temporary storage on the node-local NVMe disk.		Additionally, each compute node provides temporary storage on the node-local NVMe disk.

Registration/bwForCluster/NewRV

M Janczyk — Tue, 16 Jun 2026 09:16:06 GMT

M Janczyk deleted page Registration/bwForCluster/NewRV content was: "= Register a new RV= This page is describing step B of the bwForCluster Registration, if you are registering a new RV. To join an existing RV as a coworker, please refer back to Apply for a Rechenvorhaben/Project = Who should register an RV = Typically done only by the leader of a scientific work group or...", and the only contributor was "M Janczyk" (talk)

NEMO2/Easybuild Modules

M Janczyk — Thu, 11 Jun 2026 14:43:03 GMT

← Older revision		Revision as of 16:43, 11 June 2026
Line 73:		Line 73:
	</syntaxhighlight>		</syntaxhighlight>

	The <code>$~~ARCH~~</code> environment variable is automatically set when you load an architecture:		The <code>$MODULE_ARCH</code> environment variable is automatically set when you load an architecture:
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	module load arch/genoa		module load arch/genoa
	echo $~~ARCH~~ # Output: genoa		echo $MODULE_ARCH # Output: genoa
	</syntaxhighlight>		</syntaxhighlight>

Line 244:		Line 244:
	= Compiling Your Own Software (Non-EasyBuild) =		= Compiling Your Own Software (Non-EasyBuild) =

	If you compile your own applications outside of EasyBuild (e.g., custom C/C++/Fortran code), you should create architecture-specific builds using the <code>$~~ARCH~~</code> environment variable.		If you compile your own applications outside of EasyBuild (e.g., custom C/C++/Fortran code), you should create architecture-specific builds using the <code>$MODULE_ARCH</code> environment variable.

	== Using $~~ARCH~~ in Your Workflows ==		== Using $MODULE_ARCH in Your Workflows ==

	The <code>$~~ARCH~~</code> variable is automatically set when you load an architecture module:		The <code>$MODULE_ARCH</code> variable is automatically set when you load an architecture module:

	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	module load arch/milan		module load arch/milan
	echo $~~ARCH~~ # Output: milan		echo $MODULE_ARCH # Output: milan
	</syntaxhighlight>		</syntaxhighlight>

Line 258:		Line 258:
	<syntaxhighlight lang="bash">		<syntaxhighlight lang="bash">
	# Organize binaries by architecture		# Organize binaries by architecture
	$HOME/software/$~~ARCH~~/myapp/bin/myprogram		$HOME/software/$MODULE_ARCH/myapp/bin/myprogram

	# Example paths for different architectures:		# Example paths for different architectures:
Line 271:		Line 271:
	# Compile for Milan		# Compile for Milan
	module load arch/milan		module load arch/milan
	gcc -O3 -march=znver3 mycode.c -o $HOME/software/$~~ARCH~~/bin/myprogram		gcc -O3 -march=znver3 mycode.c -o $HOME/software/$MODULE_ARCH/bin/myprogram

	# Compile for Genoa		# Compile for Genoa
	module load arch/genoa		module load arch/genoa
	gcc -O3 -march=znver4 mycode.c -o $HOME/software/$~~ARCH~~/bin/myprogram		gcc -O3 -march=znver4 mycode.c -o $HOME/software/$MODULE_ARCH/bin/myprogram
	</syntaxhighlight>		</syntaxhighlight>

SDS@hd/Registration

H Schumacher — Tue, 09 Jun 2026 09:02:54 GMT

changed link from responsible to manager section

← Older revision		Revision as of 11:02, 9 June 2026
Line 41:		Line 41:

	If you register your own SV, you will be:		If you register your own SV, you will be:
	* ...~~responsibe~~ for providing new members with the necessary information to join the SV		* ...responsible for providing new members with the necessary information to join the SV. SV managers can do the same.
	*: → see [[SDS@hd/SV_Management#~~SV_Responsible_(SVV)~~ \| SV ~~Responsible (SVV)~~]]		*: → see [[SDS@hd/SV_Management#SV_Manager \| SV Manager]]
	* ...held accountable for the co-workers in the SV		* ...held accountable for the co-workers in the SV
	* ...asked to provide information for the two reports required by the DFG for their funding of SDS@hd		* ...asked to provide information for the two reports required by the DFG for their funding of SDS@hd

SDS@hd/SV Management

H Schumacher — Tue, 09 Jun 2026 09:00:50 GMT

moved some points from responsible to manager

← Older revision		Revision as of 11:00, 9 June 2026
(One intermediate revision by the same user not shown)
Line 5:		Line 5:
	At the [https://sds-hd.urz.uni-heidelberg.de/management SDS@hd Managementtool] members/guests can...		At the [https://sds-hd.urz.uni-heidelberg.de/management SDS@hd Managementtool] members/guests can...
	* ...view the status of your SV memberships (active/inactive)		* ...view the status of your SV memberships (active/inactive)
	* ...view the status of the SV (active/inactive, end date) (Removing people from the list completely is not possible, yet)		* ...view the status of the SV (active/inactive, end date). (Removing people from the list completely is not possible, yet.)

	=== SV Manager ===		=== SV Manager ===
	Additionally to the abilities of a coworker, a manager can...		Additionally to the abilities of a coworker, a manager can...
		⚫	* ...change the SV password
		⚫	* ...enable others to join the SV
		⚫	*# Provide them with the SV acronym. You can look it up at the [https://sds-hd.urz.uni-heidelberg.de/management/shib/info_sv.php SDS@hd Managementtool].
		⚫	*# Provide them with the SV password. The SV responsible received the initial password via email. <u>Caution</u>: Make sure to share the SV password. Do not share your personal service password which you set in registration step B at bwServices.
		⚫	*# Afterwards, they can join by following the [[SDS@hd/Registration \| registration steps]].
	* ...change the status of SV members (active/inactive)		* ...change the status of SV members (active/inactive)

	=== SV Responsible (SVV) ===		=== SV Responsible (SVV) ===
	Additionally to the abilities of a manager, the responsible can...		Additionally to the abilities of a manager, the responsible can...
⚫	* ...enable others to join the SV
⚫	*# Provide them with the SV acronym. You can look it up at the [https://sds-hd.urz.uni-heidelberg.de/management/shib/info_sv.php SDS@hd Managementtool].
⚫	*# ~~Povide~~ them with the SV password. ~~You've~~ received ~~this~~ password via email~~. You can reset it at the [https://sds-hd.urz.uni-heidelberg.de/management/shib/info_sv.php SDS@hd Managementtool]~~. <u>Caution</u>: Make sure to share the SV password ~~and~~ not your personal service password which you set in registration step 2 at bwServices.
⚫	*# Afterwards, they can join by following the [[SDS@hd/Registration \| registration steps]].
	* ...enable the guest folder		* ...enable the guest folder
	* ...change the role of SV members (guest/full member)		* ...change the role of SV members (guest/full member)
⚫	* ...change the SV password
	* ...hand over the SV responsibility		* ...hand over the SV responsibility
	* ...cancel the SV when it is not needed anymore		* ...cancel the SV when it is not needed anymore

Registration/bwForCluster/RV

M Janczyk — Wed, 03 Jun 2026 15:56:17 GMT

Become Coworker of an RV

← Older revision		Revision as of 17:56, 3 June 2026
(One intermediate revision by the same user not shown)
Line 13:		Line 13:
	== Become Coworker of an RV ==		== Become Coworker of an RV ==

			<div style="border: 3px solid #28a745; padding: 15px; background-color: #d4edda; margin: 10px 0;">
	[[File:RV-collab.png\|right\|100px\|frame\|1. RV collaboration.]]
	[[File:RV-formcol.png\|right\|300px\|thumb\|4. RV collaboration form]]
	{\|style="background:#deffee; width:70%;"
	\|style="padding:5px; background:#cef2e0; text-align:left"\|
	[[Image:Attention.svg\|center\|25px]]
	\|style="padding:5px; background:#cef2e0; text-align:left"\|
	You can only sign up for an RV once.		You can only sign up for an RV once.
	If you are an inactive member of an RV, either because you de-registered yourself or a manager deactivated you, only an RV manager can reactivate you.		If you are an inactive member of an RV, either because you de-registered yourself or a manager deactivated you, only an RV manager can reactivate you.
			</div>
	\|}

	The "RV responsible" will provide you with the following data on the RV:		The "RV responsible" will provide you with the following data on the RV:
Line 45:		Line 40:

	== Register a new RV ==		== Register a new RV ==
			<div style="border: 3px solid #28a745; padding: 15px; background-color: #d4edda; margin: 10px 0;">

		⚫	* An RV is valid for '''one year'''. After that, the RV responsible must apply for an extension of the RV for it to be renewed for another year.
	{\|style="background:#deffee; width:100%;"
			* '''Creating multiple RVs on the same cluster is not necessary.''' Being part of multiple RVs on the same cluster does not grant any additional computing resources. The resource specifications (e.g. CPU hours) in an RV are used solely for cluster (tier) routing — to help assign your project to the most suitable cluster (tier). '''They do not represent a budget that is consumed, and they do not impose any limits on your usage.''' Every cluster user has full access to all available cluster resources regardless of how many RVs they belong to.
	\|style="padding:5px; background:#cef2e0; text-align:left"\|
			</div>
	[[Image:Attention.svg\|center\|25px]]
	\|style="padding:5px; background:#cef2e0; text-align:left"\|
	An RV is valid for '''one year'''.
⚫	After that, the RV responsible must apply for an extension of the RV for it to be renewed for another year.
	\|}

	An RV should be registered by a leader or senior scientist of a research group/collaboration.		An RV should be registered by a leader or senior scientist of a research group/collaboration.
Line 66:		Line 57:
	* Any number of coworkers can join the RV to work with the cluster.Therefore, they need to be provided with the RV password and acronym.		* Any number of coworkers can join the RV to work with the cluster.Therefore, they need to be provided with the RV password and acronym.
	* The [https://www.bwhpc.de/bwhpc-domains.php research area] of the group as well as the software and hardware requirements are taken into account.		* The [https://www.bwhpc.de/bwhpc-domains.php research area] of the group as well as the software and hardware requirements are taken into account.
	*: Please look carefully through the scientific fields and research domains to find the field that fits best to your research focus and experience. This way, you'll be assigned to the cluster that can support you best. If in doubt, you can check the [https://www.dfg.de/resource/blob/175334/89ba4a3464c99aaea40fdef47367e7b2/fachsystematik-2020-2024-de-grafik-data.pdf DFG classification] as well.		*: Please look carefully through the scientific fields and research domains to find the field that fits best to your research focus and experience. This way, you'll be assigned to the cluster that can support you best. If in doubt, you can check the [https://www.dfg.de/resource/blob/175334/89ba4a3464c99aaea40fdef47367e7b2/fachsystematik-2020-2024-de-grafik-data.pdf DFG classification] as well.
	* ~~Creating~~ ~~mutiple~~ ~~RVs~~ ~~on the same cluster is~~ not necessary. Being part of multiple RVs on the same cluster does not bring any advantages regarding available computing resources. The RV description and resource specifications are important for finding the best fitting cluster. But they don't set any limits. Every cluster user has full access to ~~all cluster resources. And not all research topics must~~ match the RV description. ~~But~~ we appreciate it if you ~~update~~ the RV information ~~whenever~~ ~~you~~ ~~fill~~ in a prolongation application.		* Research topics also do not need to match the RV description exactly. However, we appreciate it if you keep the RV information up to date when submitting a prolongation application.
	* Getting access to two different clusters is only possible if specific software or hardware needs demand access for both.		* Getting access to two different clusters is only possible if specific software or hardware needs demand access for both.

Registration/bwForCluster/Helix

H Schumacher — Wed, 03 Jun 2026 15:07:33 GMT

broadened the scope of the second issue

← Older revision		Revision as of 17:07, 3 June 2026
Line 46:		Line 46:
	** No access to state services like the bwHPC clusters or SDS@hd is possible.		** No access to state services like the bwHPC clusters or SDS@hd is possible.
	** You can see that this is the issue when the login to bwServices via your identity provider is not possible.		** You can see that this is the issue when the login to bwServices via your identity provider is not possible.
	* <ins>'''Issue:</ins> Recently (re)activated uni-id:''' Your id at your home organization was (re)activated a short time ago but this change didn't get through to the bwHPC cluster or SDS@hd service.		* <ins>'''Issue:</ins> Recently (re)activated uni-id or other user data changes:''' Your id at your home organization was (re)activated a short time ago or your email address changed (or sth. similar) but this change didn't get through to the bwHPC cluster or SDS@hd service.
	** Login to bwServices. This triggers an update of your user information. Some minutes later, the connection to the service should work.		** Login to bwServices. This triggers an update of your user information. Some minutes later, the connection to the service should work again.
	* <ins>'''Issue:</ins> Entitlement / RV missing:''' Prerequisites like entitlement ownership or RV membership aren't met (anymore).		* <ins>'''Issue:</ins> Entitlement / RV missing:''' Prerequisites like entitlement ownership or RV membership aren't met (anymore).
	** You can see the concrete issue and advice on how to move forward when looking at the registry info of the service on the bwServices website.		** You can see the concrete issue and advice on how to move forward when looking at the registry info of the service on the bwServices website.

HPC Glossary/Batch system

S Fischer — Wed, 03 Jun 2026 14:25:45 GMT

Resource Management Systems on bwHPC Clusters

← Older revision		Revision as of 16:25, 3 June 2026
Line 72:		Line 72:

	Slurm: complete resource management system with integrated resource manager and scheduler		Slurm: complete resource management system with integrated resource manager and scheduler

	All bwHPC cluster follow a fairshare policy. The waiting time of jobs depends on		All bwHPC cluster follow a fairshare policy. The waiting time of jobs depends on:

	* your job's resource requests: Jobs with large resource requests wait longer for free resources.		* your job's resource requests: Jobs with large resource requests wait longer for free resources.
Line 89:		Line 90:

	FAQ: How many jobs can run on a single node at the same time?		FAQ: How many jobs can run on a single node at the same time?

	This depends on the node access policy:		This depends on the node access policy:

NEMO2/Workspaces

M Janczyk — Wed, 03 Jun 2026 12:47:43 GMT

Sharing Workspaces

← Older revision		Revision as of 14:47, 3 June 2026
Line 137:		Line 137:
	groupname: bw12a001		groupname: bw12a001
	</pre>		</pre>

			=== Share after creation ===

			If you didn't use <tt>-g</tt>/<tt>-G</tt> at creation, share read-only with <tt>ws_share</tt>:

			$ ws_share share myWs alice bob # Grant read access
			$ ws_share list myWs # Show who has access
			$ ws_share unshare myWs alice # Remove access

			'''Advanced sharing:''' [[NEMO2/Workspaces/Advanced_Features#Sharing\|Sharing guide]] for ACL-based per-user permissions.

Status

F Bartusch — Wed, 03 Jun 2026 09:34:09 GMT

← Older revision		Revision as of 11:34, 3 June 2026
Line 6:		Line 6:
	Due to work on the power supply of the cooling system all jobs have been suspended to prevent overheating.-->		Due to work on the power supply of the cooling system all jobs have been suspended to prevent overheating.-->

	{{Template:Status\|ok\|~~'''~~Status:~~'''~~ ~~Normal~~ ~~operation~~ of ~~all~~ ~~systems~~}}		{{Template:Status\|warn\|Status: BinAC 2 maintenance: June 8–12 }}

	<!--		<!--

Main Page

F Bartusch — Wed, 03 Jun 2026 09:33:32 GMT

← Older revision		Revision as of 11:33, 3 June 2026
Line 11:		Line 11:

	<!--{{Template:Status\|warn\|Status: Jobs suspended on Justus 2}}-->		<!--{{Template:Status\|warn\|Status: Jobs suspended on Justus 2}}-->
	{{Template:Status\|ok\|~~'''~~Status:~~'''~~ Ok}}		{{Template:Status\|warn\|Status: BinAC 2 maintenance: June 8–12 }}

	<!--		<!--

Data Transfer/SSHFS

K Siegmund — Wed, 03 Jun 2026 08:45:06 GMT

remove harmful otp-incompatible reconnect flags

← Older revision		Revision as of 10:45, 3 June 2026
Line 12:		Line 12:
	<li>CentOS/RedHat: <code>dnf install fuse-sshfs</code></li>		<li>CentOS/RedHat: <code>dnf install fuse-sshfs</code></li>
	<li>If you want to use /etc/fstab, you can do this with the following entry:		<li>If you want to use /etc/fstab, you can do this with the following entry:
	<pre>sshfs#<username>@<hostname>: <your_local_mountpoint> fuse defaults,user,noauto,exec~~,reconnect~~ 0 0 </pre></li></ul>		<pre>sshfs#<username>@<hostname>: <your_local_mountpoint> fuse defaults,user,noauto,exec 0 0 </pre></li></ul>

	== Usage ==		== Usage ==
Line 31:		Line 31:
	<li>Mount a folder from the cluster/storage:		<li>Mount a folder from the cluster/storage:
	<syntaxhighlight lang="bash"># Home folder		<syntaxhighlight lang="bash"># Home folder
	sshfs ~~-o reconnect~~ <username>@<hostname>: <localPath>		sshfs <username>@<hostname>: <localPath>
	# subfolderX		# subfolderX
	sshfs ~~-o reconnect~~ <username>@<hostname>:subfolderX <localPath></syntaxhighlight></li></ol>		sshfs <username>@<hostname>:subfolderX <localPath></syntaxhighlight></li></ol>

	=== Undo mounting ===		=== Undo mounting ===

Data Transfer/Rclone

H Schumacher — Tue, 02 Jun 2026 15:30:40 GMT

Added instructions for adding rclone to path on windows.

@@ Line 17: / Line 17: @@
 * Installation on [https://rclone.org/install/#windows Windows]
 * Installation on [https://rclone.org/install/#macos macOS]
 ** If Rclone shall be used together with sftp, the [[Data_Transfer/SSHFS#Installation | sshfs installation]] instructions must be followed after installing Rclone.
@@ Line 198: / Line 199: @@
 --stats-log-level=DEBUG
 </pre>

NEMO2/Containers/Enroot

M Janczyk — Tue, 02 Jun 2026 12:45:12 GMT

Image storage

← Older revision		Revision as of 14:45, 2 June 2026
Line 28:		Line 28:
	Unpacked container images are stored in <tt>~/.local/share/enroot/</tt>.		Unpacked container images are stored in <tt>~/.local/share/enroot/</tt>.
	Images are SquashFS files and can be several GB.		Images are SquashFS files and can be several GB.
	To avoid filling your home quota, store images in a [[Workspaces\|workspace]] and symlink the default path:		To avoid filling your home quota, store images in a [[NEMO2/Workspaces\|workspace]] and symlink the default path:

	<pre>		<pre>

Helix/Login

H Schumacher — Mon, 01 Jun 2026 14:04:28 GMT

Added token deletion step, changed it-service link

← Older revision		Revision as of 16:04, 1 June 2026
Line 148:		Line 148:
	'''Lost Token'''		'''Lost Token'''

	To reset your token you have to visit the IT-Service to validate your identity.		# To reset your token you have to (virtually) visit the IT Service at the University of Heidelberg to validate your identity. You can find the information for a virtual or in person visit in the "How to contact us" section here: https://www.urz.uni-heidelberg.de/en/support/it-service. It is recommended to use the Chrome browser for doing the video call.
	~~Mention~~ that you want a reset of your Helix 2FA token. It is important to emphasize that the token reset is for bwForCluster Helix to avoid confusion with the token that is specific to members/students of the university of Heidelberg.		# Please have an official identification ready (e.g. Führerschein / Personalausweis) and mention that you want a reset of your Helix 2FA token. It is important to emphasize that the token reset is for bwForCluster Helix to avoid confusion with the token that is specific to members/students of the university of Heidelberg.
			# After the IT Service deactivated the token, you can delete the token yourself, reload the page and create a new one.

		⚫	Hint: To be on the safe side, it is advised to create a [[Registration/2FA#Backup_TAN_List\|backup TAN list]] additionally to the smartphone token.
	You can visit the IT service point of the URZ in person at INF 330, 69120 Heidelberg. Or you can use the video call service. It is recommended to use the Chrome browser for doing the video call. Please have on official identification ready in any case (Führerschein / Personalausweis) .
	For further information please see here: https://it-service.uni-heidelberg.de.

⚫	To be on the safe side, it is advised to create a [[Registration/2FA#Backup_TAN_List\|backup TAN list]] additionally to the smartphone token.

Registration/HAW

R Barthel — Mon, 01 Jun 2026 07:39:30 GMT

← Older revision		Revision as of 09:39, 1 June 2026
(One intermediate revision by the same user not shown)
Line 1:		Line 1:
	The [https://hochschulen-bw.de/haw-bw/ HAW BW e.V.] is the voluntary association of the Universities of Applied Sciences Baden-Württemberg.		The [https://hochschulen-bw.de/haw-bw/ HAW BW e.V.] is the voluntary association of the Universities of Applied Sciences Baden-Württemberg.

	Currently the following colleges own a share in the bwUniCluster 2.0 and are therefore allowed to use it:		Currently the following colleges own a share in the bwUniCluster 3.0 and are therefore allowed to use it:

	* Hochschule Aalen		* Hochschule Aalen
Line 16:		Line 16:
	* All sites of the Dualen Hochschule Baden-Württemberg		* All sites of the Dualen Hochschule Baden-Württemberg

	If you want to become a shareholder of bwUniCluster 2.0, please contact the [https://www.bwhpc.de/supportportal support ticket system].		If you want to become a shareholder of bwUniCluster 3.0, please contact the [https://www.bwhpc.de/supportportal support ticket system].

	This does not apply to the bwForClusters. However, [https://www.bwhpc.de/bwhpc-domains.php research in a supported field] is needed here.		This does not apply to the bwForClusters. However, [https://www.bwhpc.de/bwhpc-domains.php research in a supported field] is needed here.

Data Transfer/Rclone

H Schumacher — Sun, 31 May 2026 13:42:21 GMT

corrected information

← Older revision		Revision as of 15:42, 31 May 2026
Line 1:		Line 1:

	[https://rclone.org/docs/ Rclone] is a command line tool to manage files on remote systems (e.g. ~~cloud~~ ~~storage~~ ~~systems).~~ ~~Rclone~~ ~~either~~ ~~synchronizes~~ in ~~one~~ ~~direction~~ ~~only or its mounting functionality is used with~~ ~~<code>rclone mount~~</~~code~~>~~. Data can be piped between two completely remote locations, sometimes without local download. One advantage is that the transfer is multithreaded and it operates on a file level basis.~~		[https://rclone.org/docs/ Rclone] is a command line tool to manage files on remote systems (e.g. high performance clusters, data and cloud storage systems, archiving services). </br>
			Rclone either synchronizes in one direction only or its mounting functionality is used with <code>rclone mount</code>. </br>
	'''Caution:''' You can't use Rclone with 2FA.
			Data can be piped between two completely remote locations. When moving data within the same system and the data transfer protocol allows for server side copy, no local download is necessary. </br>
			One advantage is that the transfer is multithreaded and it operates on a file level basis.</br>
			'''Caution:''' When using Rclone with 2FA it is necessary to connect via ssh key instead of a password.

	== Installation ==		== Installation ==
Line 8:		Line 11:

	# Download the relevant binary.		# Download the relevant binary.
	# Extract the <code>rclone</code> executable, <code>rclone.exe</code> ~~on Windows, from the archive~~.		# Extract the <code>rclone</code> executable (<code>rclone.exe</code>).
	# You can use the executables without further installation. For easy use, it is recommended to add the binary to your PATH environment variable. Information on how to do this can be found below.		# You can use the executables without further installation. For easy use, it is recommended to add the binary to your PATH environment variable. Information on how to do this can be found below.

Energy Efficient Cluster Usage

S Fischer — Fri, 29 May 2026 09:10:12 GMT

Summary: General Recommendations

← Older revision		Revision as of 11:10, 29 May 2026
Line 81:		Line 81:
	= Summary: General Recommendations =		= Summary: General Recommendations =

	* Choose the most '''efficient algorithms''' for the given problem		* Choose the most '''efficient algorithms''' for the given problem.
	* Run only '''necessary''' jobs: Please consider testing new setups and their output for validity prior to submitting a huge amount of similar jobs		* Run only '''necessary''' jobs: Please consider testing new setups and their output for validity prior to submitting a huge amount of similar jobs.
	* Start '''small''': Run ~~Your~~ problem on a small number of parallel entities (be it processes or threads) first.		* Start '''small''': Run your problem on a small number of parallel entities (be it processes or threads) first.
	* '''Estimate''' the runtime of the parallel job as '''exactly''' as possible to increase the efficiency of the scheduling of the whole system		* '''Estimate''' the runtime of the parallel job as '''exactly''' as possible to increase the efficiency of the scheduling of the whole system.
	* Use the proper tools for development: If ~~You~~ develop your own code, please use the proper tools for debugging and parallel performance analysis. More information is available on the bwHPC Wiki.		* Use the proper tools for development: If you develop your own code, please use the proper tools for debugging and parallel performance analysis. More information is available on the bwHPC Wiki.
	* A look at the '''job feedback''' can help you determine if you are using the cluster efficiently		* A look at the '''job feedback''' can help you determine if you are using the cluster efficiently.

Registration/SSH/SSH-FIDO2-Quick-Start

M Janczyk — Thu, 28 May 2026 16:26:18 GMT

← Older revision		Revision as of 18:26, 28 May 2026
Line 6:		Line 6:
	'''Why FIDO2 SSH Keys?'''		'''Why FIDO2 SSH Keys?'''

	FIDO2 SSH keys (ED25519-SK) offer significant advantages:		FIDO2 SSH keys ('''ED25519-SK''' and '''ECDSA-SK''') offer significant advantages:
	* No ~~2-factor~~ ~~unlock~~ ~~required~~ - ~~work~~ ~~immediately after~~ registration		* Valid for '''720 days''' - no frequent re-registration needed
			* No pre-login required - no need to first unlock with password and TOTP
	* Hardware-protected - private key never leaves the ~~Yubikey~~		* Hardware-protected - private key never leaves the security key
	* Physical presence required - you must touch the key to authenticate		* Physical presence required - you must touch the key to authenticate

	'''Note:''' ECDSA-SK keys are not supported on these clusters. Use ED25519-SK instead.

	{\| class="wikitable" style="text-align:center; margin-top:10px;"		{\| class="wikitable" style="text-align:center; margin-top:10px;"
Line 32:		Line 31:
	= Requirements =		= Requirements =

	* A FIDO2 security key (e.g.~~, Yubikey 5 series or newer, Feitian, Token2, etc.)~~		* A FIDO2 security key, e.g.:
			** [https://www.yubico.com/products/yubikey-5-overview/ Yubikey 5 series] or newer
			** [https://www.ftsafe.com/Products/FIDO/Single_Button_FIDO Feitian ePass FIDO A4B] (~18 €, as of May 2026) — supports <code>ecdsa-sk</code> only, use <code>ssh-keygen -t ecdsa-sk</code>
			** Token2 or any other FIDO2-certified key
	* OpenSSH 8.2 or newer		* OpenSSH 8.2 or newer
	* For PIN setup: Either Chrome/Chromium/Brave browser OR <code>yubikey-manager</code> command-line tool (Yubikey only)		* For PIN setup: Either Chrome/Chromium/Brave browser OR <code>yubikey-manager</code> command-line tool (Yubikey only)
Line 40:		Line 42:
	<pre>		<pre>
	ssh -V		ssh -V
	ssh -Q PubkeyAcceptedAlgorithms \| grep sk-ssh-ed25519~~@openssh.com~~		ssh -Q PubkeyAcceptedAlgorithms \| grep -E 'sk-ssh-ed25519\|sk-ecdsa'
	</pre>		</pre>

Line 119:		Line 121:

	<div style="border: 3px solid #17a2b8; padding: 15px; background-color: #d1ecf1; margin: 10px 0;">		<div style="border: 3px solid #17a2b8; padding: 15px; background-color: #d1ecf1; margin: 10px 0;">
	'''FIDO2 SSH Keys work immediately ~~without~~ 2-~~factor~~ ~~authentication~~!'''		'''FIDO2 SSH Keys work immediately - no pre-login required!'''

	Unlike regular SSH keys, FIDO2 keys do not need to be "unlocked" with ~~username/~~password.		Unlike regular SSH keys, FIDO2 keys do not need to be unlocked by first logging in with password and TOTP.
	They work as soon as you register them as '''Interactive Keys'''.		They work as soon as you register them as '''Interactive Keys''' and are valid for '''720 days'''.
	</div>		</div>

Line 132:		Line 134:
	</pre>		</pre>

	2. Copy the complete output (starts with <code>sk-ssh-ed25519@openssh.com</code>)		2. Copy the complete output (starts with <code>sk-ssh-ed25519@openssh.com</code> for ED25519-SK, or <code>sk-ecdsa-sha2-nistp256@openssh.com</code> for ECDSA-SK)

	3. Follow the [[Registration/SSH#Adding_a_new_SSH_Key\|SSH Key Registration Guide]] to:		3. Follow the [[Registration/SSH#Adding_a_new_SSH_Key\|SSH Key Registration Guide]] to:

Registration/SSH

M Janczyk — Thu, 28 May 2026 16:12:33 GMT

← Older revision		Revision as of 18:12, 28 May 2026
Line 12:		Line 12:
	\| bwUniCluster 3.0		\| bwUniCluster 3.0
	\| style="background-color:#90EE90;" \| bwIDM Portal		\| style="background-color:#90EE90;" \| bwIDM Portal
	\| Centralized key management, 180-day validity		\| Centralized key management, 180-day validity (720-day for FIDO2 SK keys)
	\|-		\|-
	\| bwForCluster BinAC 2		\| bwForCluster BinAC 2
Line 28:		Line 28:
	\| bwForCluster NEMO 2		\| bwForCluster NEMO 2
	\| style="background-color:#90EE90;" \| bwIDM Portal		\| style="background-color:#90EE90;" \| bwIDM Portal
	\| Centralized key management, 180-day validity		\| Centralized key management, 180-day validity (720-day for FIDO2 SK keys)
	\|}		\|}

Line 103:		Line 103:
	Centralized SSH key management provides:		Centralized SSH key management provides:

	* Security enforcement: Keys must use strong algorithms ~~and~~ have 180-day validity		* Security enforcement: Keys must use strong algorithms; regular keys have 180-day validity, FIDO2 SK keys have 720-day validity
	* Centralized control: Review and revoke all keys from one location		* Centralized control: Review and revoke all keys from one location
	* Two key types: Interactive keys (manual logins) and Command keys (automated workflows)		* Two key types: Interactive keys (manual logins) and Command keys (automated workflows)
Line 121:		Line 121:
	=== FIDO2 Hardware Keys ===		=== FIDO2 Hardware Keys ===

	'''ED25519-SK''' keys use hardware security keys (like Yubikey) for authentication:		'''ED25519-SK''' and '''ECDSA-SK''' keys use hardware security keys (like Yubikey or Feitian ePass FIDO A4B) for authentication:

			* Valid for '''720 days''' - no frequent re-registration needed
	* Always valid - no 2-factor unlock required
			* No pre-login required - no need to first unlock with password and TOTP
	* Hardware-protected - private key never leaves the device		* Hardware-protected - private key never leaves the device
	* Physical presence required - must touch key to authenticate		* Physical presence required - must touch key to authenticate

	'''Note:''' ECDSA-SK keys are not supported. Use ED25519-SK only.

	{\| class="wikitable" style="text-align:center;"		{\| class="wikitable" style="text-align:center;"
Line 133:		Line 132:
	\|-		\|-
	! Cluster		! Cluster
	! ED25519-SK ~~Support~~		! SK Support (ED25519-SK / ECDSA-SK)
	\|-		\|-
	\| bwUniCluster 3.0		\| bwUniCluster 3.0
Line 151:		Line 150:
	<div style="border: 3px solid #ffc107; background-color: #fff3cd; margin: 10px 0px 10px 0px;">		<div style="border: 3px solid #ffc107; background-color: #fff3cd; margin: 10px 0px 10px 0px;">
	<p style="padding-left: 5px; margin: 0px;">'''Important:'''</p>		<p style="padding-left: 5px; margin: 0px;">'''Important:'''</p>
	* ~~Keys~~ are valid for '''180 days''' and automatically revoked after expiration.		* Regular SSH keys are valid for '''180 days''' and automatically revoked after expiration. FIDO2 hardware keys (SK) are valid for '''720 days'''.
	* Upload only your '''public key''' file (ending in <code>.pub</code>, e.g., <code>~/.ssh/id_ed25519.pub</code>).		* Upload only your '''public key''' file (ending in <code>.pub</code>, e.g., <code>~/.ssh/id_ed25519.pub</code>).
	</div>		</div>
Line 211:		Line 210:
	\|}		\|}

	'''FIDO2 Hardware Keys''' (ED25519-SK):		'''FIDO2 Hardware Keys''' (ED25519-SK, ECDSA-SK):
	* ~~Always~~ ~~valid~~ - no 2-~~factor unlock~~ needed		* Valid for '''720 days''' - no re-authentication needed
			* No pre-login required - no password/TOTP unlock needed
	* Authentication via physical key touch only		* Authentication via physical key touch only
	* Only on bwUniCluster 3.0 and NEMO 2 (not Helix)		* Only on bwUniCluster 3.0 and NEMO 2 (not Helix)
Line 256:		Line 256:
	* IP restriction: Limit to specific IP address(es) or subnet		* IP restriction: Limit to specific IP address(es) or subnet
	* Admin approval: Keys require review before activation		* Admin approval: Keys require review before activation
	* Short validity: Maximum 30 days
	</div>		</div>

SDS@hd/Access

H Schumacher — Wed, 27 May 2026 16:06:37 GMT

added 'domain' as additional bullet point to avoid confusion

← Older revision		Revision as of 18:06, 27 May 2026
Line 15:		Line 15:
	* '''Hostname:''' The hostname depends on the chosen network protocol:		* '''Hostname:''' The hostname depends on the chosen network protocol:
	** For [[Data_Transfer/SSHFS\|SSHFS]] and [[Data_Transfer/SFTP\|SFTP]]: ''lsdf02-sshfs.urz.uni-heidelberg.de''		** For [[Data_Transfer/SSHFS\|SSHFS]] and [[Data_Transfer/SFTP\|SFTP]]: ''lsdf02-sshfs.urz.uni-heidelberg.de''
	** For [[SDS@hd/Access/SMB\|SMB]] and [[SDS@hd/Access/NFS\|NFS]]: ''lsdf02.urz.uni-heidelberg.de''~~. The domain is ''BWSERVICESAD''.~~		** For [[SDS@hd/Access/SMB\|SMB]] and [[SDS@hd/Access/NFS\|NFS]]: ''lsdf02.urz.uni-heidelberg.de''
	** For [[Data_Transfer/WebDAV\|WebDAV]] the url is: ''https://lsdf02-webdav.urz.uni-heidelberg.de''		** For [[Data_Transfer/WebDAV\|WebDAV]] the url is: ''https://lsdf02-webdav.urz.uni-heidelberg.de''
			* '''Domain:''' The domain is ''BWSERVICESAD'' (needed for smb and nfs connections).

	== Recommended Setup ==		== Recommended Setup ==

Data Transfer/Rclone

H Schumacher — Wed, 27 May 2026 16:01:30 GMT

improved Windows instructions

← Older revision		Revision as of 18:01, 27 May 2026
(One intermediate revision by the same user not shown)
Line 15:		Line 15:
	* Installation on [https://rclone.org/install/#windows Windows]		* Installation on [https://rclone.org/install/#windows Windows]
	* Installation on [https://rclone.org/install/#macos macOS]		* Installation on [https://rclone.org/install/#macos macOS]
	** If Rclone shall be used together with sftp, the [[~~https://wiki.bwhpc.de/e/~~Data_Transfer/SSHFS#Installation \| sshfs installation]] instructions must be followed after installing Rclone.		** If Rclone shall be used together with sftp, the [[Data_Transfer/SSHFS#Installation \| sshfs installation]] instructions must be followed after installing Rclone.
	* Installation on [https://rclone.org/install/#script-installation Linux]		* Installation on [https://rclone.org/install/#script-installation Linux]

Line 135:		Line 135:
	=== Windows ===		=== Windows ===

	To run rclone mount on Windows, you will need to [https://winfsp.dev/rel/ download and install WinFsp]. ~~To mount on drive letter X or a nonexistent subdirectory, use:~~		To run rclone mount on Windows, you will need to [https://winfsp.dev/rel/ download and install WinFsp] first.

			<syntaxhighlight lang="python"># mount under drive letter X (or any other free letter)
	<pre>rclone mount <remote-name>:path/to/files X:
	rclone mount ~~<~~remote-name~~>~~:path/to/files C:~~\path\parent\mount</pre>~~		rclone mount --vfs-cache-mode full <remote-name>:<path/to/remote/files> X:
			# or provide the path to a nonexisting folder:
			rclone mount --vfs-cache-mode full <remote-name>:<path/to/remote/files> <C:\path\parentDir\nonexistingFoldername></syntaxhighlight>
			To unmount, the command can be stopped in the terminal with the key combination CTRL+C. Alternatively, a mounted drive can be unmounted by making a right click on the drive name in the file explorer and choose "Eject". </br>
	In contrast to Linux/Mac, there is no background mode.		In contrast to Linux/Mac, there is no background mode.

Line 147:		Line 150:
	Create an empty directory on your local machine and then execute		Create an empty directory on your local machine and then execute

	<~~pre~~># to mount the root folder:		<syntaxhighlight lang="bash"># to mount the root folder:
	rclone mount --vfs-cache-mode full ~~<~~remote-name~~>~~: /path/to/empty/folder		rclone mount --vfs-cache-mode full <remote-name>: /path/to/empty/folder
	# to mount a subfolder:		# to mount a subfolder:
	rclone mount --vfs-cache-mode full ~~<~~remote-name~~>~~:folderX/folderY /path/to/empty/folder		rclone mount --vfs-cache-mode full <remote-name>:folderX/folderY /path/to/empty/folder
	# to unmount:		# to unmount:
	fusermount -uz /path/to/mounted/folder </~~pre~~>		fusermount -uz /path/to/mounted/folder </syntaxhighlight>

	== Best Practices ==		== Best Practices ==