BwUniCluster2.0/Batch Queues - Revision history

S Braun: /* Partitions, Queues and Jobs */

2025-01-17T13:27:10Z

Partitions, Queues and Jobs

← Older revision		Revision as of 15:27, 17 January 2025
Line 29:		Line 29:

	For interactive jobs, the resources are requested with the ''salloc'' command. As soon as the computing resources are available and allocated, a command line prompt is returned on a computing node and the user can freely dispose of the resources now available to him.		For interactive jobs, the resources are requested with the ''salloc'' command. As soon as the computing resources are available and allocated, a command line prompt is returned on a computing node and the user can freely dispose of the resources now available to him.
			{\|style="background:#deffee; width:100%;"

			\|style="padding:5px; background:#cef2e0; text-align:left"\|

			[[Image:Attention.svg\|center\|25px]]

			\|style="padding:5px; background:#cef2e0; text-align:left"\|
	'''Please remember:'''		'''Please remember:'''
	* Heavy computations are not allowed on the login nodes. Use a developement or a regular job queue instead! Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].		* Heavy computations are not allowed on the login nodes.<br>Use a developement or a regular job queue instead! Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].
	* Development queues are meant for development tasks. Do not misuse this queue for regular, short-running jobs or chain jobs! Only one running job at a time is enabled.		* Development queues are meant for development tasks.<br>Do not misuse this queue for regular, short-running jobs or chain jobs! Only one running job at a time is enabled.
			\|}

	== Batch Jobs: sbatch ==		== Batch Jobs: sbatch ==

S Braun: /* Partitions, Queues and Jobs */

2025-01-17T13:21:20Z

Partitions, Queues and Jobs

← Older revision		Revision as of 15:21, 17 January 2025
Line 34:		Line 34:
	'''Please remember:'''		'''Please remember:'''
	* Heavy computations are not allowed on the login nodes. Use a developement or a regular job queue instead! Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].		* Heavy computations are not allowed on the login nodes. Use a developement or a regular job queue instead! Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].
	* Development queues are meant for development tasks. Do not misuse this queue for regular, short-running jobs or chain jobs!		* Development queues are meant for development tasks. Do not misuse this queue for regular, short-running jobs or chain jobs! Only one running job at a time is enabled.

	== Batch Jobs: sbatch ==		== Batch Jobs: sbatch ==

S Braun: /* Partitions, Queues and Jobs */

2025-01-17T13:14:12Z

Partitions, Queues and Jobs

← Older revision		Revision as of 15:14, 17 January 2025
Line 34:		Line 34:
	'''Please remember:'''		'''Please remember:'''
	* Heavy computations are not allowed on the login nodes. Use a developement or a regular job queue instead! Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].		* Heavy computations are not allowed on the login nodes. Use a developement or a regular job queue instead! Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].
	* Development queues are meant for development tasks. Do not ~~use~~ this queue for regular short-running jobs or chain jobs.		* Development queues are meant for development tasks. Do not misuse this queue for regular, short-running jobs or chain jobs!

	== Batch Jobs: sbatch ==		== Batch Jobs: sbatch ==

S Braun: /* Partitions, Queues and Jobs */

2025-01-17T13:12:53Z

Partitions, Queues and Jobs

← Older revision		Revision as of 15:12, 17 January 2025
Line 20:		Line 20:
	** multiple: Jobs that request more than one node.		** multiple: Jobs that request more than one node.
	** gpu: Jobs that request GPU accelerators on one or more than one node.		** gpu: Jobs that request GPU accelerators on one or more than one node.
	* Development queues		* Development queues (dev)
	** ~~dev:~~ Short, usually interactive jobs that are used for developing, compiling and testing code and workflows. The intention behind development queues is to provide users with immediate access to computer resources without having to wait. This is the place to realize instantaneous heavy compute without affecting other users, as would be the case on the login nodes.		** Short, usually interactive jobs that are used for developing, compiling and testing code and workflows. The intention behind development queues is to provide users with immediate access to computer resources without having to wait. This is the place to realize instantaneous heavy compute without affecting other users, as would be the case on the login nodes.

	Requested compute resources such as (wall-)time, number of nodes and amount of memory are restricted and must fit into the boundaries imposed by the queues. The request for compute resources on the bwUniCluster 2.0 <font color=red>requires at least the specification of the '''queue''' and the '''time'''</font>.		Requested compute resources such as (wall-)time, number of nodes and amount of memory are restricted and must fit into the boundaries imposed by the queues. The request for compute resources on the bwUniCluster 2.0 <font color=red>requires at least the specification of the '''queue''' and the '''time'''</font>.

S Braun: /* Partitions, Queues and Jobs */

2025-01-17T13:11:56Z

Partitions, Queues and Jobs

← Older revision		Revision as of 15:11, 17 January 2025
Line 21:		Line 21:
	** gpu: Jobs that request GPU accelerators on one or more than one node.		** gpu: Jobs that request GPU accelerators on one or more than one node.
	* Development queues		* Development queues
	** dev: Short, usually interactive jobs that are used for developing, compiling and testing code and workflows.		** dev: Short, usually interactive jobs that are used for developing, compiling and testing code and workflows. The intention behind development queues is to provide users with immediate access to computer resources without having to wait. This is the place to realize instantaneous heavy compute without affecting other users, as would be the case on the login nodes.

	Requested compute resources such as (wall-)time, number of nodes and amount of memory are restricted and must fit into the boundaries imposed by the queues. The request for compute resources on the bwUniCluster 2.0 <font color=red>requires at least the specification of the '''queue''' and the '''time'''</font>.		Requested compute resources such as (wall-)time, number of nodes and amount of memory are restricted and must fit into the boundaries imposed by the queues. The request for compute resources on the bwUniCluster 2.0 <font color=red>requires at least the specification of the '''queue''' and the '''time'''</font>.

S Braun at 13:07, 17 January 2025

2025-01-17T13:07:19Z

← Older revision		Revision as of 15:07, 17 January 2025
Line 29:		Line 29:
	For interactive jobs, the resources are requested with the ''salloc'' command. As soon as the computing resources are available and allocated, a command line prompt is returned on a computing node and the user can freely dispose of the resources now available to him.		For interactive jobs, the resources are requested with the ''salloc'' command. As soon as the computing resources are available and allocated, a command line prompt is returned on a computing node and the user can freely dispose of the resources now available to him.

	Since heavy computations on the login nodes are not allowed, there are also two so-called development queues which can be used for short interactive jobs and use cases like building a particularly big piece of software.

⚫	== sbatch -p ''queue'' ==

			'''Please remember:'''
			* Heavy computations are not allowed on the login nodes. Use a developement or a regular job queue instead! Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].
			* Development queues are meant for development tasks. Do not use this queue for regular short-running jobs or chain jobs.

			== Batch Jobs: sbatch ==

		⚫	sbatch -p ''queue''

	<br>		<br>
Line 135:		Line 140:
	<br>		<br>

	== Interactive Jobs ==		== Interactive Jobs: salloc ==

	On bwUniCluster 2.0 you are only allowed to run short jobs (<< 1 hour) with little memory requirements (<< 8 GByte) on the logins nodes. If you want to run longer jobs and/or jobs with a request of more than 8 GByte of memory, you must allocate resources for so-called interactive jobs by usage of the command salloc on a login node. Considering a serial application running on a compute node that requires 5000 MByte of memory and limiting the interactive run to 2 hours the following command has to be executed:		On bwUniCluster 2.0 you are only allowed to run short jobs (<< 1 hour) with little memory requirements (<< 8 GByte) on the logins nodes. If you want to run longer jobs and/or jobs with a request of more than 8 GByte of memory, you must allocate resources for so-called interactive jobs by usage of the command salloc on a login node. Considering a serial application running on a compute node that requires 5000 MByte of memory and limiting the interactive run to 2 hours the following command has to be executed:

S Braun: /* Partitions, Queues and Jobs */

2025-01-17T13:00:48Z

Partitions, Queues and Jobs

← Older revision		Revision as of 15:00, 17 January 2025
Line 22:		Line 22:
	* Development queues		* Development queues
	** dev: Short, usually interactive jobs that are used for developing, compiling and testing code and workflows.		** dev: Short, usually interactive jobs that are used for developing, compiling and testing code and workflows.
	Requested compute resources such as (wall-)time, number of nodes and amount of memory are restricted and must fit into the boundaries imposed by the queues. The request for compute resources on the bwUniCluster 2.0 <font color=red>requires at least the specification of the queue and the time</font>.		Requested compute resources such as (wall-)time, number of nodes and amount of memory are restricted and must fit into the boundaries imposed by the queues. The request for compute resources on the bwUniCluster 2.0 <font color=red>requires at least the specification of the '''queue''' and the '''time'''</font>.

	Jobs can be run non-interactively as '''batch jobs''' or as '''interactive jobs'''.		Jobs can be run non-interactively as '''batch jobs''' or as '''interactive jobs'''.
	Submitting a batch job means, that all steps of a compute project are defined in a bash script. This bash script is queued and executed as soon as the compute resources are allocated.		Submitting a batch job means, that all steps of a compute project are defined in a bash script. This bash script is queued and executed as soon as the compute resources are available and allocated. Jobs are enqueued with the ''sbatch'' command.

			For interactive jobs, the resources are requested with the ''salloc'' command. As soon as the computing resources are available and allocated, a command line prompt is returned on a computing node and the user can freely dispose of the resources now available to him.

	Since heavy computations on the login nodes are not allowed, there are also two so-called development queues which can be used for short interactive jobs and use cases like building a particularly big piece of software.		Since heavy computations on the login nodes are not allowed, there are also two so-called development queues which can be used for short interactive jobs and use cases like building a particularly big piece of software.

S Braun: /* Partitions, Queues and Jobs */

2025-01-17T12:49:52Z

Partitions, Queues and Jobs

← Older revision		Revision as of 14:49, 17 January 2025
Line 25:		Line 25:

	Jobs can be run non-interactively as '''batch jobs''' or as '''interactive jobs'''.		Jobs can be run non-interactively as '''batch jobs''' or as '''interactive jobs'''.
			Submitting a batch job means, that all steps of a compute project are defined in a bash script. This bash script is queued and executed as soon as the compute resources are allocated.

	Since heavy computations on the login nodes are not allowed, there are also two so-called development queues which can be used for short interactive jobs and use cases like building a particularly big piece of software.		Since heavy computations on the login nodes are not allowed, there are also two so-called development queues which can be used for short interactive jobs and use cases like building a particularly big piece of software.


	== sbatch -p ''queue'' ==		== sbatch -p ''queue'' ==

S Braun at 12:45, 17 January 2025

2025-01-17T12:45:42Z

← Older revision		Revision as of 14:45, 17 January 2025
Line 22:		Line 22:
	* Development queues		* Development queues
	** dev: Short, usually interactive jobs that are used for developing, compiling and testing code and workflows.		** dev: Short, usually interactive jobs that are used for developing, compiling and testing code and workflows.
		⚫	Requested compute resources such as (wall-)time, number of nodes and amount of memory are restricted and must fit into the boundaries imposed by the queues. The request for compute resources on the bwUniCluster 2.0 <font color=red>requires at least the specification of the queue and the time</font>.

			Jobs can be run non-interactively as '''batch jobs''' or as '''interactive jobs'''.

⚫	Requested compute resources such as (wall-)time, number of nodes and amount of memory are restricted and must fit into the boundaries imposed by the queues.

	Since heavy computations on the login nodes are not allowed, there are also two so-called development queues which can be used for short interactive jobs and use cases like building a particularly big piece of software.		Since heavy computations on the login nodes are not allowed, there are also two so-called development queues which can be used for short interactive jobs and use cases like building a particularly big piece of software.



	Since requested compute resources are NOT always automatically mapped to the correct queue class, '''you must add the correct queue class to your sbatch command '''. <font color=red>The specification of a queue is obligatory on bwUniCluster 2.0.</font>

	== sbatch -p ''queue'' ==		== sbatch -p ''queue'' ==

S Braun: /* Partitions, Queues and Jobs */

2025-01-17T12:35:16Z

Partitions, Queues and Jobs

← Older revision		Revision as of 14:35, 17 January 2025
Line 21:		Line 21:
	** gpu: Jobs that request GPU accelerators on one or more than one node.		** gpu: Jobs that request GPU accelerators on one or more than one node.
	* Development queues		* Development queues
	** dev: ~~For short~~, ~~in most cases~~ interactive, jobs, that are used for ~~development~~, ~~heavy~~ compiling and testing of code and workflows.		** dev: Short, usually interactive jobs that are used for developing, compiling and testing code and workflows.


			Requested compute resources such as (wall-)time, number of nodes and amount of memory are restricted and must fit into the boundaries imposed by the queues.
	group similar node types (e.g. nodes with and without accelerators) and to enforce different access policies and resource limits.

	Since heavy computations on the login nodes are not allowed, there are also two so-called development queues which can be used for short interactive jobs and use cases like building a particularly big piece of software.		Since heavy computations on the login nodes are not allowed, there are also two so-called development queues which can be used for short interactive jobs and use cases like building a particularly big piece of software.



	~~Compute resources such as (wall-)time, nodes and memory are restricted and must fit into '''queues'''.~~ Since requested compute resources are NOT always automatically mapped to the correct queue class, '''you must add the correct queue class to your sbatch command '''. <font color=red>The specification of a queue is obligatory on bwUniCluster 2.0.</font>		Since requested compute resources are NOT always automatically mapped to the correct queue class, '''you must add the correct queue class to your sbatch command '''. <font color=red>The specification of a queue is obligatory on bwUniCluster 2.0.</font>

	== sbatch -p ''queue'' ==		== sbatch -p ''queue'' ==