Running Calculations - Revision history

K Siegmund: /* Link to Batch System per Cluster */

2026-03-25T09:09:52Z

Link to Batch System per Cluster

@@ Line 25: / Line 25: @@
 &rarr; '''[[Environment Modules]]'''
-== Link to Batch System per Cluster ==
+== Job Script and Job Submission ==
 Because of differences in configuration (partly due to different available hardware), each cluster has their own batch system documentation:

A Flachmueller: /* Life Cycle of a Calculation (Job) */

2026-03-11T11:46:16Z

Life Cycle of a Calculation (Job)

← Older revision		Revision as of 13:46, 11 March 2026
Line 6:		Line 6:
	# prepare a script (a set commands to run - usually as a shell script), with all the commands that are necessary to run your calculation from start to finish. In addition to the commands necessary to run the calculation, this ''[[batch script]]'' has a header section, in which you specify details like required compute cores (processing units witin a computer), estimated runtime, memory requirements, disk space needed, etc.		# prepare a script (a set commands to run - usually as a shell script), with all the commands that are necessary to run your calculation from start to finish. In addition to the commands necessary to run the calculation, this ''[[batch script]]'' has a header section, in which you specify details like required compute cores (processing units witin a computer), estimated runtime, memory requirements, disk space needed, etc.
	# ''Submit'' the script into a queue, where your ''job'' (calculation)		# ''Submit'' the script into a queue, where your ''job'' (calculation)
	# is queued and waits in row with other compute jobs until the resources you requested in the header become available.		# gets asigned an inital priority, is queued and waits in row with other compute jobs until the resources you requested in the header become available. (Requested time is also a resource!)
	# Execution: Once a suitable resource slot is available and your job is in the front of the queue of suitable jobs (suitable with regards to the resource slot), your script is executed on (a) compute node(s). Your calculation runs on that/those node(s) until it is finished or reaches the specified time limit.		# Execution: Once a suitable resource slot is available and your job is in the front of the queue of suitable jobs (suitable with regards to the resource slot), your script is executed on (a) compute node(s). Your calculation runs on that/those node(s) until it is finished or reaches the specified time limit.
	# Save results: Include commands to save the calculation results back to a long term storage (e.g. your home directory), at least at the end of your script. What you have not saved until the job finishes won't be saved!		# Save results: Include commands to save the calculation results back to a long term storage (e.g. your home directory), at least at the end of your script. What you have not saved until the job finishes won't be saved!

A Flachmueller: /* Life Cycle of a Calculation (Job) */

2026-03-11T11:15:34Z

Life Cycle of a Calculation (Job)

← Older revision		Revision as of 13:15, 11 March 2026
Line 8:		Line 8:
	# is queued and waits in row with other compute jobs until the resources you requested in the header become available.		# is queued and waits in row with other compute jobs until the resources you requested in the header become available.
	# Execution: Once a suitable resource slot is available and your job is in the front of the queue of suitable jobs (suitable with regards to the resource slot), your script is executed on (a) compute node(s). Your calculation runs on that/those node(s) until it is finished or reaches the specified time limit.		# Execution: Once a suitable resource slot is available and your job is in the front of the queue of suitable jobs (suitable with regards to the resource slot), your script is executed on (a) compute node(s). Your calculation runs on that/those node(s) until it is finished or reaches the specified time limit.
	# Save results: Include commands to save the calculation results back to a long term storage (e.g. your home directory), at least at the end of your script.		# Save results: Include commands to save the calculation results back to a long term storage (e.g. your home directory), at least at the end of your script. What you have not saved until the job finishes won't be saved!
			# If your job reaches the specified time limit, all your running processes will be killed and the resources get cleared. So any data that has not been saved will be lost!

	The software that distributes jobs on compute nodes is called a '''[[batch system]]''' or '''batch scheduler'''. The software currently used as a [[batch system]] on bwHPC clusters is "Slurm".		The software that distributes jobs on compute nodes is called a '''[[batch system]]''' or '''batch scheduler'''. The software currently used as a [[batch system]] on bwHPC clusters is "Slurm".

A Flachmueller: Execution step is expressed more broadly + minor corrections

2026-03-10T16:34:04Z

Execution step is expressed more broadly + minor corrections

M Janczyk: M Janczyk moved page HPC Glossary/Running Calculations to Running Calculations over redirect

2025-07-15T09:30:53Z

M Janczyk moved page HPC Glossary/Running Calculations to Running Calculations over redirect

← Older revision	Revision as of 11:30, 15 July 2025
(No difference)

M Janczyk: M Janczyk moved page Running Calculations to HPC Glossary/Running Calculations

2025-07-15T09:28:28Z

M Janczyk moved page Running Calculations to HPC Glossary/Running Calculations

← Older revision	Revision as of 11:28, 15 July 2025
(No difference)

K Siegmund: /* Example Jobs */

2025-07-15T09:22:36Z

Example Jobs

← Older revision		Revision as of 11:22, 15 July 2025
Line 22:		Line 22:
	How to access these examples is described in the "Software job examples" section of the page		How to access these examples is described in the "Software job examples" section of the page

	→ '''[[~~Environment_Modules~~]]'''		→ '''[[Environment Modules]]'''

	== Link to Batch System per Cluster ==		== Link to Batch System per Cluster ==

K Siegmund at 09:13, 15 July 2025

2025-07-15T09:13:42Z

← Older revision		Revision as of 11:13, 15 July 2025
Line 35:		Line 35:

	→ '''[[NEMO2/Slurm \| Slurm NEMO2]]'''		→ '''[[NEMO2/Slurm \| Slurm NEMO2]]'''

			→ '''[[BinAC2/Slurm \| Slurm BinAC2]]'''

	== How to Use Computing Ressources Efficiently ==		== How to Use Computing Ressources Efficiently ==

K Siegmund: /* How to Use Computing Ressources Efficiently */

2025-07-15T09:11:29Z

How to Use Computing Ressources Efficiently

← Older revision		Revision as of 11:11, 15 July 2025
Line 45:		Line 45:

	→ '''[[Scaling]]'''		→ '''[[Scaling]]'''


	Running calculations on an HPC node consumes a lot of energy. To make the most of the available resources and keep cluster and energy use as efficient as possible please also see our advice for		Running calculations on an HPC node consumes a lot of energy. To make the most of the available resources and keep cluster and energy use as efficient as possible please also see our advice for

K Siegmund: /* Link to Batch System per Cluster */

2025-07-15T09:11:20Z

Link to Batch System per Cluster

← Older revision		Revision as of 11:11, 15 July 2025
Line 27:		Line 27:

	Because of differences in configuration (partly due to different available hardware), each cluster has their own batch system documention:		Because of differences in configuration (partly due to different available hardware), each cluster has their own batch system documention:

	* Slurm systems

	→ '''[[BwUniCluster3.0/Running_Jobs\|Slurm bwUniCluster 3.0]]'''		→ '''[[BwUniCluster3.0/Running_Jobs\|Slurm bwUniCluster 3.0]]'''
Line 37:		Line 35:

	→ '''[[NEMO2/Slurm \| Slurm NEMO2]]'''		→ '''[[NEMO2/Slurm \| Slurm NEMO2]]'''

	* Moab systems (legacy systems with deprecated queuing system)
	→ '''[[BinAC/Moab\|Moab BinAC]]'''

	== How to Use Computing Ressources Efficiently ==		== How to Use Computing Ressources Efficiently ==

← Older revision		Revision as of 18:34, 10 March 2026
Line 2:		Line 2:
	== Life Cycle of a Calculation (Job) ==		== Life Cycle of a Calculation (Job) ==
	[[File:running_calculations_on_cluster.svg\|thumb\|upright=0.4]]		[[File:running_calculations_on_cluster.svg\|thumb\|upright=0.4]]
	On your desktop computer, you start your calculations and they start immediately, ~~run~~ until they are finished, ~~then~~ your desktop does mostly nothing, until you start another calculation. A compute cluster has several hundred, maybe a thousand computers (compute nodes), all of them are busy most of the time and many people want to run a great number of calculations. So running your job has to include some extra steps:		On your desktop computer you start your calculations and they start immediately, running until they are finished. Then your desktop does mostly nothing, until you start another calculation. A compute cluster has several hundred, maybe a thousand computers (compute nodes), all of them are busy most of the time and many people want to run a great number of calculations. So running your job has to include some extra steps:

	# prepare a script (a set commands to run - usually as a shell script), with all the commands that are necessary to run your calculation from start to finish. In addition to the commands necessary to run the calculation, this ''[[batch script]]'' has a header section, in which you specify details like required compute cores (processing units witin a computer), estimated runtime, memory requirements, disk space needed, etc.		# prepare a script (a set commands to run - usually as a shell script), with all the commands that are necessary to run your calculation from start to finish. In addition to the commands necessary to run the calculation, this ''[[batch script]]'' has a header section, in which you specify details like required compute cores (processing units witin a computer), estimated runtime, memory requirements, disk space needed, etc.
	# ''Submit'' the script into a queue, where your ''job'' (calculation)		# ''Submit'' the script into a queue, where your ''job'' (calculation)
	# is queued and waits in row with other compute jobs until the resources you requested in the header become available.		# is queued and waits in row with other compute jobs until the resources you requested in the header become available.
	# Execution: Once your job ~~reaches~~ the front of the queue, your script is executed on a compute node. Your calculation runs on that node until it is finished or reaches the specified time limit.		# Execution: Once a suitable resource slot is available and your job is in the front of the queue of suitable jobs (suitable with regards to the resource slot), your script is executed on (a) compute node(s). Your calculation runs on that/those node(s) until it is finished or reaches the specified time limit.
	# Save results: ~~At the end of your script, include~~ commands to save the calculation results back to your home directory.		# Save results: Include commands to save the calculation results back to a long term storage (e.g. your home directory), at least at the end of your script.

	The software that distributes ~~Jobs~~ on compute nodes is called a '''[[batch system]]''' or '''batch scheduler'''. The software currently used as a [[batch system]] on bwHPC clusters is "Slurm".		The software that distributes jobs on compute nodes is called a '''[[batch system]]''' or '''batch scheduler'''. The software currently used as a [[batch system]] on bwHPC clusters is "Slurm".

	Learn more about the functioning of job distribution in		Learn more about the functioning of job distribution in
Line 26:		Line 26:
	== Link to Batch System per Cluster ==		== Link to Batch System per Cluster ==

	Because of differences in configuration (partly due to different available hardware), each cluster has their own batch system ~~documention~~:		Because of differences in configuration (partly due to different available hardware), each cluster has their own batch system documentation:

	→ '''[[BwUniCluster3.0/Running_Jobs\|Slurm bwUniCluster 3.0]]'''		→ '''[[BwUniCluster3.0/Running_Jobs\|Slurm bwUniCluster 3.0]]'''
Line 41:		Line 41:


	When you are running your calculations, you will have to decide on how many compute-cores your ~~calculation~~ will be simultaneously calculated.		When you are running your calculations, you will have to decide on how many compute-cores your job will be simultaneously calculated.
	For this, your computational problem will have to be divided into pieces, which always causes some overhead.		For this, your computational problem will have to be divided into pieces, which always causes some overhead.