BwUniCluster3.0/Hardware and Architecture - Revision history

S Braun: /* Compute nodes */

2026-07-20T07:47:38Z

Compute nodes

← Older revision		Revision as of 09:47, 20 July 2026
Line 69:		Line 69:
	\| <code>cpu</code>, <code>dev_cpu</code>		\| <code>cpu</code>, <code>dev_cpu</code>
	\| <code>highmem</code>, <code>dev_highmem</code>		\| <code>highmem</code>, <code>dev_highmem</code>
	\| <code>gpu_h100</code>, <code>dev_gpu_h100</code>		\| <code>gpu_h100</code>, <code>dev_gpu_h100</code>, <code>gpu_h100_short</code>
	\| <code>gpu_mi300</code>		\| <code>gpu_mi300</code>
	\| <code>gpu_a100_il</code> / <code>gpu_h100_il</code>		\| <code>gpu_a100_il</code> / <code>gpu_h100_il</code>
Line 79:		Line 79:
	\| 80		\| 80
	\| 5		\| 5
	\| 12		\| 19
	\| 1		\| 1
	\| 15		\| 15

S Braun: /* Compute nodes */

2026-05-19T06:43:21Z

Compute nodes

← Older revision		Revision as of 08:43, 19 May 2026
Line 151:		Line 151:
	\| 94 GiB		\| 94 GiB
	\| APU		\| APU
	\| 80 ~~GiB / 94~~ GiB		\| 80 GiB
	\| 40 GiB		\| 40 GiB
	\| -		\| -

S Braun: /* Compute nodes */

2026-04-21T10:59:26Z

Compute nodes

← Older revision		Revision as of 12:59, 21 April 2026
Line 78:		Line 78:
	\| 272		\| 272
	\| 80		\| 80
	\| 4		\| 5
	\| 12		\| 12
	\| 1		\| 1

S Braun: /* Compute nodes */

2026-04-21T10:58:43Z

Compute nodes

← Older revision		Revision as of 12:58, 21 April 2026
Line 77:		Line 77:
	!scope="column"\| Number of nodes		!scope="column"\| Number of nodes
	\| 272		\| 272
	\| 70		\| 80
	\| 4		\| 4
	\| 12		\| 12

R Barthel at 22:45, 3 September 2025

2025-09-03T22:45:34Z

← Older revision		Revision as of 00:45, 4 September 2025
Line 149:		Line 149:
	\| -		\| -
	\| -		\| -
	\| 94 GB		\| 94 GiB
	\| APU		\| APU
	\| 80 GB / 94 GB		\| 80 GiB / 94 GiB
	\| 40 GB		\| 40 GiB
	\| -		\| -
	\|-		\|-

R Barthel at 22:44, 3 September 2025

2025-09-03T22:44:22Z

← Older revision		Revision as of 00:44, 4 September 2025
Line 116:		Line 116:
	\|-		\|-
	!scope="column"\| Main memory		!scope="column"\| Main memory
	\| 256 GB		\| 256 GiB
	\| 384 GB		\| 384 GiB
	\| ~~2.3~~ TB		\| 2304 GiB
	\| 768 GB		\| 768 GiB
	\| 4x 128 GB HBM3		\| 4x 128 GiB HBM3
	\| 512 GB		\| 512 GiB
	\| 384 GB		\| 384 GiB
	\| 384 GB		\| 384 GiB
	\|-		\|-
	!scope="column"\| Local SSD		!scope="column"\| Local SSD

S Braun: /* Compute nodes */

2025-09-02T09:45:21Z

Compute nodes

← Older revision		Revision as of 11:45, 2 September 2025
Line 52:		Line 52:
	* '''AMD GPU x4''': AMD's accelerated processing unit (APU) MI300A with 4 CPU sockets and 4 compute units which share the same high-bandwidth memory (HBM).		* '''AMD GPU x4''': AMD's accelerated processing unit (APU) MI300A with 4 CPU sockets and 4 compute units which share the same high-bandwidth memory (HBM).
	* '''Ice Lake NVIDIA GPU x4''': Similar to the Ice Lake nodes, but with larger memory and four NVIDIA A100 or H100 GPUs.		* '''Ice Lake NVIDIA GPU x4''': Similar to the Ice Lake nodes, but with larger memory and four NVIDIA A100 or H100 GPUs.
			* '''Cascade Lake NVIDIA GPU x4''': Nodes with four NVIDIA A100 GPUs.
	{\| class="wikitable"		{\| class="wikitable"
	\|-		\|-
Line 61:		Line 62:
	! style="width:10%"\| GPU node<br/>AMD GPU x4		! style="width:10%"\| GPU node<br/>AMD GPU x4
	! style="width:10%"\| GPU nodes<br/>Ice Lake<br/>NVIDIA GPU x4		! style="width:10%"\| GPU nodes<br/>Ice Lake<br/>NVIDIA GPU x4
			! style="width:10%"\| GPU nodes<br/>Cascade Lake<br/>NVIDIA GPU x4
	! style="width:10%"\| Login nodes		! style="width:10%"\| Login nodes
	\|-		\|-
Line 70:		Line 72:
	\| <code>gpu_mi300</code>		\| <code>gpu_mi300</code>
	\| <code>gpu_a100_il</code> / <code>gpu_h100_il</code>		\| <code>gpu_a100_il</code> / <code>gpu_h100_il</code>
			\| <code>gpu_a100_short</code></code>
	\| -		\| -
	\|-		\|-
Line 79:		Line 82:
	\| 1		\| 1
	\| 15		\| 15
			\| 19
	\| 2		\| 2
	\|-		\|-
Line 88:		Line 92:
	\| AMD Zen 4		\| AMD Zen 4
	\| Intel Xeon Platinum 8358		\| Intel Xeon Platinum 8358
			\| Intel Xeon Gold 6248R
	\| AMD EPYC 9454		\| AMD EPYC 9454
	\|-		\|-
Line 98:		Line 103:
	\| 2		\| 2
	\| 2		\| 2
	\|-		\| 2
	!scope="column"\| Processor frequency (GHz)
	\| 2.6 GHz
	\| 2.75 GHz
	\| 2.75 GHz
	\| 2.75 GHz
	\| 3.7 GHz
	\| 2.6 GHz
	\| 2.75 GHz
	\|-		\|-
	!scope="column"\| Total number of cores		!scope="column"\| Total number of cores
Line 115:		Line 112:
	\| 96 (4x 24)		\| 96 (4x 24)
	\| 64		\| 64
			\| 48
	\| 96		\| 96
	\|-		\|-
Line 124:		Line 122:
	\| 4x 128 GB HBM3		\| 4x 128 GB HBM3
	\| 512 GB		\| 512 GB
			\| 384 GB
	\| 384 GB		\| 384 GB
	\|-		\|-
Line 132:		Line 131:
	\| 15.36 TB NVMe		\| 15.36 TB NVMe
	\| 7.68 TB NVMe		\| 7.68 TB NVMe
	\| 6.4 TB NVMe		\| 6.4 TB NVMe
			\| 1.92 TB SATA SSD
	\| 7.68 TB SATA SSD		\| 7.68 TB SATA SSD
	\|-		\|-
Line 142:		Line 142:
	\| 4x AMD Instinct MI300A		\| 4x AMD Instinct MI300A
	\| 4x NVIDIA A100 / H100		\| 4x NVIDIA A100 / H100
			\| 4x NVIDIA A100
	\| -		\| -
	\|-		\|-
Line 150:		Line 151:
	\| 94 GB		\| 94 GB
	\| APU		\| APU
	\| 80 GB / 94 GB		\| 80 GB / 94 GB
			\| 40 GB
	\| -		\| -
	\|-		\|-
Line 160:		Line 162:
	\| IB 2x NDR200		\| IB 2x NDR200
	\| IB 2x HDR200		\| IB 2x HDR200
			\| IB 4x EDR
	\| IB 1x NDR200		\| IB 1x NDR200
	\|}		\|}

S Braun: /* File Systems */

2025-04-06T17:20:26Z

File Systems

← Older revision		Revision as of 19:20, 6 April 2025
Line 189:		Line 189:
	several months.		several months.

	For further details please check		For further details please check: [[BwUniCluster3.0/Hardware_and_Architecture/Filesystem_Details\|File System Details]]

	[[BwUniCluster3.0/Hardware_and_Architecture/Filesystem_Details\|File System Details]]

	== $HOME ==		== $HOME ==

S Braun: /* File Systems */

2025-04-06T17:18:47Z

File Systems

← Older revision		Revision as of 19:18, 6 April 2025
Line 177:		Line 177:
	'''Which file system to use?'''		'''Which file system to use?'''

	~~In general, you~~ should separate your data and store it on the appropriate file system.		You should separate your data and store it on the appropriate file system.
	Permanently needed data like software or important results should be stored ~~below~~ $HOME		Permanently needed data like software or important results should be stored in $HOME but capacity restrictions (quotas) apply.
	~~but capacity restrictions (quotas) apply.~~ In case you accidentally deleted data on $HOME		In case you accidentally deleted data on $HOME there is a chance that we can restore it from backup.
			Permanent data which is not needed for months or exceeds the capacity restrictions should be sent to the LSDF Online Storage or to the archive and deleted from the file systems. Temporary data which is only needed on a single node and which does not exceed the disk space shown in Table 1 above should be stored
	there is a chance that we can restore it from backup. Permanent data which is not needed for months
	or exceeds the capacity restrictions should be sent to the LSDF Online Storage
	or to the archive and deleted from the file systems. Temporary data which is only needed on a single
	node and which does not exceed the disk space shown in Table 1 above should be stored
	below $TMPDIR. Data which is read many times on a single node, e.g. if you are doing AI training,		below $TMPDIR. Data which is read many times on a single node, e.g. if you are doing AI training,
	should be copied to $TMPDIR and read from there. Temporary data which is used from many nodes		should be copied to $TMPDIR and read from there. Temporary data which is used from many nodes

S Braun: /* Architecture of bwUniCluster 3.0 */

2025-04-06T17:14:55Z

Architecture of bwUniCluster 3.0

← Older revision		Revision as of 19:14, 6 April 2025
Line 13:		Line 13:
	The individual nodes of the system act in different roles. From an end users point of view the different groups of nodes are login nodes and compute nodes. File server nodes and administrative server nodes are not accessible by users.		The individual nodes of the system act in different roles. From an end users point of view the different groups of nodes are login nodes and compute nodes. File server nodes and administrative server nodes are not accessible by users.

	'''Login Nodes'''		'''Login Nodes'''<br>

	The login nodes are the only nodes directly accessible by end users. These nodes are used for interactive login, file management, program development, and interactive pre- and post-processing.		The login nodes are the only nodes directly accessible by end users. These nodes are used for interactive login, file management, program development, and interactive pre- and post-processing.
	There are two nodes dedicated to this service, but they can all be reached from a single address: <code>uc3.scc.kit.edu</code>. A DNS round-robin alias distributes login sessions to the login nodes.		There are two nodes dedicated to this service, but they can all be reached from a single address: <code>uc3.scc.kit.edu</code>. A DNS round-robin alias distributes login sessions to the login nodes.
	To prevent login nodes from being used for activities that are not permitted there and that affect the user experience of other users, '''long-running and/or compute-intensive tasks are periodically terminated without any prior warning'''. Please refer to [[BwUniCluster3.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].		To prevent login nodes from being used for activities that are not permitted there and that affect the user experience of other users, '''long-running and/or compute-intensive tasks are periodically terminated without any prior warning'''. Please refer to [[BwUniCluster3.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].

	'''Compute Nodes'''		'''Compute Nodes'''<br>

	The majority of nodes are compute nodes which are managed by a batch system. Users submit their jobs to the SLURM batch system and a job is executed when the required resources become available (depending on its fair-share priority).		The majority of nodes are compute nodes which are managed by a batch system. Users submit their jobs to the SLURM batch system and a job is executed when the required resources become available (depending on its fair-share priority).

	'''File Systems'''		'''File Systems'''<br>
	bwUniCluster 3.0 comprises two parallel file systems based on Lustre.		bwUniCluster 3.0 comprises two parallel file systems based on Lustre.