BinAC2/SLURM Partitions - Revision history

S Behnle: Added BinAC2 long partition

2025-09-18T07:42:40Z

Added BinAC2 long partition

← Older revision		Revision as of 09:42, 18 September 2025
Line 1:		Line 1:
	== Partitions ==		== Partitions ==

	The bwForCluster BinAC 2 provides ~~two~~ partitions for job submission.		The bwForCluster BinAC 2 provides four partitions for job submission.
	Within a partition job allocations are routed automatically to the most suitable compute node(s) for the requested resources (e.g. amount of nodes and cores, memory, number and type of GPUs).		Within a partition job allocations are routed automatically to the most suitable compute node(s) for the requested resources (e.g. amount of nodes and cores, memory, number and type of GPUs).

	The <code>gpu</code> partition will only run 8 jobs per user at the same time. A user can only use 4 A100 and 8 A30 GPUs at the same time.		The <code>gpu</code> partition will only run 8 jobs per user at the same time. A user can only use 4 A100 and 8 A30 GPUs at the same time.

	The <code>interactive</code> will only run 1 job per user at the same time.		The <code>interactive</code> partition will only run 1 job per user at the same time.
	This partition is reserved is dedicated for testing things and using tools via a graphical user ~~interfae~~.		This partition is reserved is dedicated for testing things and using tools via a graphical user interface.
	The four nodes <code>node1-00[1-4]</code> are exclusively reserved for this partition.		The four nodes <code>node1-00[1-4]</code> are exclusively reserved for this partition.
	You can run a VNC server in this partition. Please use <code>#SBATCH --gres=display:1</code> in your jobscript or <code>--gres=display:1</code> on the command line if you need a display. This ensures that your job starts on a node with "free" displays, because each of the four nodes only provide 20 possible virtual displays.		You can run a VNC server in this partition. Please use <code>#SBATCH --gres=display:1</code> in your jobscript or <code>--gres=display:1</code> on the command line if you need a display. This ensures that your job starts on a node with "free" displays, because each of the four nodes only provide 20 possible virtual displays.

			The <code>long</code> partition is meant for long-running, parallel jobs. Please pack your jobs as dense as possible. If possible, do regular checkpointing in case the job fails after several days. Due to the small number of GPU nodes at BinAC2, we cannot offer a <code>long</code> partition with GPU nodes.

	<!--		<!--
Line 41:		Line 43:
	\| ntasks=1, time=00:10:00, mem-per-cpu=1gb		\| ntasks=1, time=00:10:00, mem-per-cpu=1gb
	\| time=10:00:00</br>MaxJobsPerUser: 1		\| time=10:00:00</br>MaxJobsPerUser: 1
			\|-
			\| long
			\| shared
			\| cpu (InfiniBand nodes only)
			\| time=1-00:00:00, feature=ib
			\| time=30-00:00:00</br>MaxNodes=10
	\|-		\|-
	\|}		\|}

F Bartusch at 12:38, 20 August 2025

2025-08-20T12:38:09Z

← Older revision		Revision as of 14:38, 20 August 2025
Line 9:		Line 9:
	This partition is reserved is dedicated for testing things and using tools via a graphical user interfae.		This partition is reserved is dedicated for testing things and using tools via a graphical user interfae.
	The four nodes <code>node1-00[1-4]</code> are exclusively reserved for this partition.		The four nodes <code>node1-00[1-4]</code> are exclusively reserved for this partition.
			You can run a VNC server in this partition. Please use <code>#SBATCH --gres=display:1</code> in your jobscript or <code>--gres=display:1</code> on the command line if you need a display. This ensures that your job starts on a node with "free" displays, because each of the four nodes only provide 20 possible virtual displays.

	<!--		<!--

F Bartusch at 12:34, 20 August 2025

2025-08-20T12:34:03Z

← Older revision		Revision as of 14:34, 20 August 2025
Line 5:		Line 5:

	The <code>gpu</code> partition will only run 8 jobs per user at the same time. A user can only use 4 A100 and 8 A30 GPUs at the same time.		The <code>gpu</code> partition will only run 8 jobs per user at the same time. A user can only use 4 A100 and 8 A30 GPUs at the same time.

			The <code>interactive</code> will only run 1 job per user at the same time.
			This partition is reserved is dedicated for testing things and using tools via a graphical user interfae.
			The four nodes <code>node1-00[1-4]</code> are exclusively reserved for this partition.

	<!--		<!--
	All partitions are operated in shared mode, that is, jobs from different users can be executed on the same node. However, one can get exclusive access to compute nodes by using the "--exclusive" option.		All partitions are operated in shared mode, that is, jobs from different users can be executed on the same node. However, one can get exclusive access to compute nodes by using the "--exclusive" option.
Line 29:		Line 34:
	gres/gpu:a30=8,		gres/gpu:a30=8,
	gres/gpu:h200=4</pre>		gres/gpu:h200=4</pre>
			\|-
			\| interactive
			\| shared
			\| cpu
			\| ntasks=1, time=00:10:00, mem-per-cpu=1gb
			\| time=10:00:00</br>MaxJobsPerUser: 1
	\|-		\|-
	\|}		\|}

F Bartusch at 16:48, 24 June 2025

2025-06-24T16:48:44Z

← Older revision		Revision as of 18:48, 24 June 2025
Line 26:		Line 26:
	\| gpu		\| gpu
	\| ntasks=1, time=00:10:00, mem-per-cpu=1gb		\| ntasks=1, time=00:10:00, mem-per-cpu=1gb
	\| time=14-00:00:00</br>MaxJobsPerUser: 8</br>MaxTRESPerUser: <~~code~~>gres/gpu:a100=4,~~gres/gpu:a30=8</code>~~		\| time=14-00:00:00</br>MaxJobsPerUser: 8</br>MaxTRESPerUser:</br><pre>gres/gpu:a100=4,
			gres/gpu:a30=8,
			gres/gpu:h200=4</pre>
	\|-		\|-
	\|}		\|}
Line 54:		Line 56:
	\| 4		\| 4
	\| <code>--gres=gpu:a100:N</code>		\| <code>--gres=gpu:a100:N</code>
			\|-
			\| Nvidia H200
			\| 141GB
			\| 4
			\| <code>--gres=gpu:h200:N</code>
	\|-		\|-
	\|}		\|}

F Bartusch at 10:13, 24 June 2025

2025-06-24T10:13:55Z

← Older revision		Revision as of 12:13, 24 June 2025
Line 1:		Line 1:
	== Partitions ==		== Partitions ==

	The bwForCluster BinAC 2 provides two partitions ~~(e.g. queues)~~ for job submission.		The bwForCluster BinAC 2 provides two partitions for job submission.
	Within a partition job allocations are routed automatically to the most suitable compute node(s) for the requested resources (e.g. amount of nodes and cores, memory, number of GPUs).		Within a partition job allocations are routed automatically to the most suitable compute node(s) for the requested resources (e.g. amount of nodes and cores, memory, number and type of GPUs).

	The <code>gpu</code> partition will only run 8 jobs per user at the same time. A user can only use 4 A100 and 8 A30 GPUs at the same time.		The <code>gpu</code> partition will only run 8 jobs per user at the same time. A user can only use 4 A100 and 8 A30 GPUs at the same time.

F Bartusch at 14:20, 19 December 2024

2024-12-19T14:20:45Z

← Older revision		Revision as of 16:20, 19 December 2024
Line 1:		Line 1:
	== Partitions ==		== Partitions ==

	The bwForCluster BinAC 2 provides two partitions (e.g. queues) for job submission. ~~Within a partition job allocations are routed automatically to the most suitable compute node(s) for the requested resources (e.g. amount of nodes and cores, memory, number of GPUs).~~		The bwForCluster BinAC 2 provides two partitions (e.g. queues) for job submission.
			Within a partition job allocations are routed automatically to the most suitable compute node(s) for the requested resources (e.g. amount of nodes and cores, memory, number of GPUs).

		⚫	The <code>gpu</code> partition will only run 8 jobs per user at the same time. A user can only use 4 A100 and 8 A30 GPUs at the same time.
	<!--		<!--
	All partitions are operated in shared mode, that is, jobs from different users can be executed on the same node. However, one can get exclusive access to compute nodes by using the "--exclusive" option.		All partitions are operated in shared mode, that is, jobs from different users can be executed on the same node. However, one can get exclusive access to compute nodes by using the "--exclusive" option.
Line 7:		Line 10:
	{\| class="wikitable"		{\| class="wikitable"
	\|-		\|-
	! style="width:20%"\| Partition		! style="width:10%"\| Partition
	! style="width:20%"\| Node Access Policy		! style="width:10%"\| Node Access Policy
	! style="width:20%"\| Node Types		! style="width:10%"\| Node Types
	! style="width:20%"\| Default		! style="width:20%"\| Default
	! style="width:20%"\| Limits		! style="width:20%"\| Limits
Line 23:		Line 26:
	\| gpu		\| gpu
	\| ntasks=1, time=00:10:00, mem-per-cpu=1gb		\| ntasks=1, time=00:10:00, mem-per-cpu=1gb
			\| time=14-00:00:00</br>MaxJobsPerUser: 8</br>MaxTRESPerUser: <code>gres/gpu:a100=4,gres/gpu:a30=8</code>
	\| nodes=1, time=14-00:00:00
	\|-		\|-
	\|}		\|}

	=== Quality of Service (QOS) ===

	A Quality of Service (QOS) limits the resources a user or group can use at the same time. Usually the QoS has the same name as the partitio it is associated with. You can list all QoS via:

	<pre>
	$ sacctmgr list qos --parsable
	Name\|Priority\|GraceTime\|Preempt\|PreemptExemptTime\|PreemptMode\|Flags\|UsageThres\|UsageFactor\|GrpTRES\|GrpTRESMins\|GrpTRESRunMins\|GrpJobs\|GrpSubmit\|GrpWall\|MaxTRES\|MaxTRESPerNode\|MaxTRESMins\|MaxWall\|MaxTRESPU\|MaxJobsPU\|MaxSubmitPU\|MaxTRESPA\|MaxTRESRunMinsPA\|MaxTRESRunMinsPU\|MaxJobsPA\|MaxSubmitPA\|MinTRES\|
	normal\|0\|00:00:00\|\|\|cluster\|\|\|1.000000\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|
	gpu\|0\|00:00:00\|\|\|cluster\|\|\|1.000000\|\|\|\|\|\|\|\|\|\|\|gres/gpu:a100=4,gres/gpu:a30=8\|8\|\|\|\|\|\|\|\|
	</pre>

⚫	The ~~output shows that the~~ <code>gpu</code> partition will only run 8 jobs per user at the same time. A user can only use 4 A100 and 8 A30 GPUs at the same time~~. QoS may change in the future, please check the current values with the <code>sacctmgr list qos</code> command on the cluster~~.

	=== Parallel Jobs ===		=== Parallel Jobs ===

F Bartusch at 14:07, 19 December 2024

2024-12-19T14:07:40Z

← Older revision		Revision as of 16:07, 19 December 2024
Line 27:		Line 27:
	\|}		\|}

			=== Quality of Service (QOS) ===

			A Quality of Service (QOS) limits the resources a user or group can use at the same time. Usually the QoS has the same name as the partitio it is associated with. You can list all QoS via:

			<pre>
			$ sacctmgr list qos --parsable
			Name\|Priority\|GraceTime\|Preempt\|PreemptExemptTime\|PreemptMode\|Flags\|UsageThres\|UsageFactor\|GrpTRES\|GrpTRESMins\|GrpTRESRunMins\|GrpJobs\|GrpSubmit\|GrpWall\|MaxTRES\|MaxTRESPerNode\|MaxTRESMins\|MaxWall\|MaxTRESPU\|MaxJobsPU\|MaxSubmitPU\|MaxTRESPA\|MaxTRESRunMinsPA\|MaxTRESRunMinsPU\|MaxJobsPA\|MaxSubmitPA\|MinTRES\|
			normal\|0\|00:00:00\|\|\|cluster\|\|\|1.000000\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|\|
			gpu\|0\|00:00:00\|\|\|cluster\|\|\|1.000000\|\|\|\|\|\|\|\|\|\|\|gres/gpu:a100=4,gres/gpu:a30=8\|8\|\|\|\|\|\|\|\|
			</pre>

			The output shows that the <code>gpu</code> partition will only run 8 jobs per user at the same time. A user can only use 4 A100 and 8 A30 GPUs at the same time. QoS may change in the future, please check the current values with the <code>sacctmgr list qos</code> command on the cluster.

	=== Parallel Jobs ===		=== Parallel Jobs ===

F Bartusch at 15:12, 4 December 2024

2024-12-04T15:12:56Z

← Older revision		Revision as of 17:12, 4 December 2024
Line 34:		Line 34:
	=== GPU Jobs ===		=== GPU Jobs ===

	BinAC 2 provides different GPU models for computations. Please select the appropriate GPU type ~~via~~ the <code>--gres=aXX:~~1..~~N</code> option in your job script		BinAC 2 provides different GPU models for computations. Please select the appropriate GPU type and the amount of GPUs with the <code>--gres=aXX:N</code> option in your job script

	{\| class="wikitable"		{\| class="wikitable"
Line 46:		Line 46:
	\| 24GB		\| 24GB
	\| 2		\| 2
	\| <code>--gres=gpu:a30:~~[1..~~N]</code>		\| <code>--gres=gpu:a30:N</code>
	\|-		\|-
	\| Nvidia A100		\| Nvidia A100
	\| 80GB		\| 80GB
	\| 4		\| 4
	\| <code>--gres=gpu:a100:~~[1..~~N]</code>		\| <code>--gres=gpu:a100:N</code>
	\|-		\|-
	\|}		\|}

F Bartusch at 13:36, 4 December 2024

2024-12-04T13:36:36Z

← Older revision		Revision as of 15:36, 4 December 2024
Line 1:		Line 1:
	== Partitions ==		== Partitions ==

	On bwForCluster BinAC 2 it is ~~necessary~~ to ~~request~~ ~~a partition with <code>--partition=<partition_name></code> on~~ job submission. Within a partition job allocations are routed automatically to the most suitable compute node(s) for the requested resources (e.g. amount of nodes and cores, memory, number of GPUs)~~. The devel partition is the default partition, if no partition is requested~~.		The bwForCluster BinAC 2 provides two partitions (e.g. queues) for job submission. Within a partition job allocations are routed automatically to the most suitable compute node(s) for the requested resources (e.g. amount of nodes and cores, memory, number of GPUs).



	<!--		<!--
	~~The~~ partitions ~~devel, cpu-single and gpu-single~~ are operated in shared mode, ~~i.e.~~ jobs from different users can ~~run~~ on the same node. ~~Jobs~~ can get exclusive access to compute nodes ~~in these partitions~~ ~~with~~ the "--exclusive" option~~. The partitions cpu-multi and gpu-multi are operated in exclusive mode. Jobs in these partitions automatically get exclusive access to the requested compute nodes~~.		All partitions are operated in shared mode, that is, jobs from different users can be executed on the same node. However, one can get exclusive access to compute nodes by using the "--exclusive" option.

	GPUs are requested with the option "--gres=gpu:<number-of-gpus>".
	-->		-->

	{\| class="wikitable"		{\| class="wikitable"
	\|-		\|-
Line 19:		Line 13:
	! style="width:20%"\| Limits		! style="width:20%"\| Limits
	\|-		\|-
			\| compute (default)
	\| eth
	\| shared		\| shared
	\| cpu		\| cpu
	\| ntasks=1, time=00:10:00, mem-per-cpu=~~2gb~~		\| ntasks=1, time=00:10:00, mem-per-cpu=1gb
	\| nodes=2, time=00:30:00		\| nodes=2, time=14-00:00:00
	\|-		\|-
	\| ib		\| gpu
	\| shared		\| shared
	\| ~~cpu~~		\| gpu
	\| ntasks=1, time=00:30:00, mem-per-cpu=~~2gb~~		\| ntasks=1, time=00:10:00, mem-per-cpu=1gb
	\| nodes=1, time=~~120~~:00:00		\| nodes=1, time=14-00:00:00
			\|-
			\|}


			=== Parallel Jobs ===

			In order to submit parallel jobs to the InfiniBand part of the cluster, i.e., for fast inter-node communication, please select the appropriate nodes via the <code>--constraint=ib</code> option in your job script. For less demanding parallel jobs, you may try the <code>--constraint=eth</code> option, which utilizes 100Gb/s Ethernet instead of the low-latency 100Gb/s InfiniBand.

			=== GPU Jobs ===

			BinAC 2 provides different GPU models for computations. Please select the appropriate GPU type via the <code>--gres=aXX:1..N</code> option in your job script

			{\| class="wikitable"
			\|-
			! style="width:20%"\| GPU
			! style="width:20%"\| GPU Memory
			! style="width:20%"\| # GPUs per Node [N]
			! style="width:20%"\| Submit Option
			\|-
			\| Nvidia A30
			\| 24GB
			\| 2
			\| <code>--gres=gpu:a30:[1..N]</code>
			\|-
			\| Nvidia A100
			\| 80GB
			\| 4
			\| <code>--gres=gpu:a100:[1..N]</code>
	\|-		\|-
	\|}		\|}

F Bartusch: Create stub

2024-12-03T15:45:23Z

Create stub

New page

== Partitions ==

On bwForCluster BinAC 2 it is necessary to request a partition with <code>--partition=<partition_name></code> on job submission. Within a partition job allocations are routed automatically to the most suitable compute node(s) for the requested resources (e.g. amount of nodes and cores, memory, number of GPUs). The devel partition is the default partition, if no partition is requested.



{| class="wikitable"
|-
! style="width:20%"| Partition
! style="width:20%"| Node Access Policy
! style="width:20%"| Node Types
! style="width:20%"| Default
! style="width:20%"| Limits
|-
| eth
| shared
| cpu
| ntasks=1, time=00:10:00, mem-per-cpu=2gb
| nodes=2, time=00:30:00
|-
| ib
| shared
| cpu
| ntasks=1, time=00:30:00, mem-per-cpu=2gb
| nodes=1, time=120:00:00
|-
|}