BwUniCluster2.0/Hardware and Architecture - Revision history

S Braun: /* Components of bwUniCluster */

2025-01-17T10:38:38Z

Components of bwUniCluster

← Older revision		Revision as of 12:38, 17 January 2025
Line 44:		Line 44:
	! style="width:13%"\| Compute nodes "Thin"		! style="width:13%"\| Compute nodes "Thin"
	! style="width:13%"\| Compute nodes "HPC"		! style="width:13%"\| Compute nodes "HPC"
	! style="width:13%"\| Compute nodes "~~IceLake~~"		! style="width:13%"\| Compute nodes "Ice Lake"
	! style="width:13%"\| Compute nodes "Fat"		! style="width:13%"\| Compute nodes "Fat"
	! style="width:13%"\| GPU x4		! style="width:13%"\| GPU x4
	! style="width:13%"\| GPU x8		! style="width:13%"\| GPU x8
	! style="width:13%"\| ~~IceLake~~ + GPU x4		! style="width:13%"\| Ice Lake + GPU x4
	! style="width:13%"\| Login		! style="width:13%"\| Login
	\|-		\|-

S Braun: /* Architecture of bwUniCluster 2.0 */

2025-01-17T09:56:32Z

Architecture of bwUniCluster 2.0

← Older revision		Revision as of 11:56, 17 January 2025
Line 20:		Line 20:
	different login nodes.		different login nodes.

	To prevent login nodes from being used for activities that are not permitted there and that affect ~~other~~ ~~users~~ in ~~their~~ ~~user~~ ~~experience~~, '''long-running and/or compute-intensive tasks are terminated ~~regularly and~~ without any prior warning'''. Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].		To prevent login nodes from being used for activities that are not permitted there and that affect the user experience of other users, '''long-running and/or compute-intensive tasks are periodically terminated without any prior warning'''. Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].

	'''Compute Node'''		'''Compute Node'''

S Braun: /* Architecture of bwUniCluster 2.0 */

2025-01-17T09:55:05Z

Architecture of bwUniCluster 2.0

← Older revision		Revision as of 11:55, 17 January 2025
Line 20:		Line 20:
	different login nodes.		different login nodes.

	To prevent login nodes from being used for activities that are not permitted there and that affect other users in their user experience, long-running and compute-intensive tasks are terminated regularly and ~~with~~ any prior warning. Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].		To prevent login nodes from being used for activities that are not permitted there and that affect other users in their user experience, '''long-running and/or compute-intensive tasks are terminated regularly and without any prior warning'''. Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].

	'''Compute Node'''		'''Compute Node'''

S Braun: /* Architecture of bwUniCluster 2.0 */

2025-01-17T09:54:01Z

Architecture of bwUniCluster 2.0

← Older revision		Revision as of 11:54, 17 January 2025
Line 19:		Line 19:
	one address and a DNS round-robin alias distributes the login sessions to the		one address and a DNS round-robin alias distributes the login sessions to the
	different login nodes.		different login nodes.

			To prevent login nodes from being used for activities that are not permitted there and that affect other users in their user experience, long-running and compute-intensive tasks are terminated regularly and with any prior warning. Please refer to [[BwUniCluster2.0/Login#Allowed_Activities_on_Login_Nodes\|Allowed Activities on Login Nodes]].

	'''Compute Node'''		'''Compute Node'''

R Laifer: /* $TMPDIR */

2024-11-26T12:08:20Z

$TMPDIR

← Older revision		Revision as of 14:08, 26 November 2024
Line 467:		Line 467:
	[[Image:Attention.svg\|center\|25px]]		[[Image:Attention.svg\|center\|25px]]
	\|style="padding:5px; background:#f2cece; text-align:left"\|		\|style="padding:5px; background:#f2cece; text-align:left"\|
	Note that you should '''not''' use /tmp or /scratch! Please ~~and~~ use '''$TMPDIR''' instead.<br/>		Note that you should '''not''' use /tmp or /scratch! Please use '''$TMPDIR''' instead.<br/>
	The reason is that an automatic cleanup on /tmp or /scratch is not possible because another job could be still using data below these directories. Hence the corresponding file systems could fill up and this can cause issues for you and for other users. On the other hand, $TMPDIR is created when the job starts and removed when the job completes, i.e. a cleanup is automatically done.		The reason is that an automatic cleanup on /tmp or /scratch is not possible because another job could be still using data below these directories. Hence the corresponding file systems could fill up and this can cause issues for you and for other users. On the other hand, $TMPDIR is created when the job starts and removed when the job completes, i.e. a cleanup is automatically done.
	\|}		\|}

S Braun: /* Use $TMPDIR instead of /tmp or /scratch */

2024-11-26T10:06:01Z

Use $TMPDIR instead of /tmp or /scratch

@@ Line 504: / Line 504: @@
 rsync -av $TMPDIR/results $(ws_find data-ssd)/results-${SLURM_JOB_ID}/
 </source>
 == LSDF Online Storage==

S Braun: /* $TMPDIR */

2024-11-26T10:05:50Z

$TMPDIR

← Older revision		Revision as of 12:05, 26 November 2024
Line 462:		Line 462:
	compiled and linked in a subdirectory of $TMPDIR. The real installation of the package (e.g. make install)		compiled and linked in a subdirectory of $TMPDIR. The real installation of the package (e.g. make install)
	should be made into the $HOME folder.		should be made into the $HOME folder.

			{\|style="background:#ffdeee; width:100%;"
			\|style="padding:5px; background:#f2cece; text-align:left"\|
			[[Image:Attention.svg\|center\|25px]]
			\|style="padding:5px; background:#f2cece; text-align:left"\|
			Note that you should '''not''' use /tmp or /scratch! Please and use '''$TMPDIR''' instead.<br/>
			The reason is that an automatic cleanup on /tmp or /scratch is not possible because another job could be still using data below these directories. Hence the corresponding file systems could fill up and this can cause issues for you and for other users. On the other hand, $TMPDIR is created when the job starts and removed when the job completes, i.e. a cleanup is automatically done.
			\|}


	=== Usage example for $TMPDIR ===		=== Usage example for $TMPDIR ===

S Braun: /* Use $TMPDIR instead of /tmp or /scratch */

2024-11-26T10:04:57Z

Use $TMPDIR instead of /tmp or /scratch

← Older revision		Revision as of 12:04, 26 November 2024
Line 498:		Line 498:
	=== Use $TMPDIR instead of /tmp or /scratch ===		=== Use $TMPDIR instead of /tmp or /scratch ===


⚫	~~Note that you should ''not'' use /tmp or /scratch and use $TMPDIR instead.~~ The reason is that an automatic cleanup on /tmp or /scratch is not possible because another job could be still using data below these directories. Hence the corresponding file systems could fill up and this can cause issues for you and for other users. On the other hand, $TMPDIR is created when the job starts and removed when the job completes, i.e. a cleanup is automatically done.


			{\|style="background:#ffdeee; width:100%;"
			\|style="padding:5px; background:#f2cece; text-align:left"\|
			[[Image:Attention.svg\|center\|25px]]
			\|style="padding:5px; background:#f2cece; text-align:left"\|
			Note that you should '''not''' use /tmp or /scratch! Please and use '''$TMPDIR''' instead.<br/>
		⚫	The reason is that an automatic cleanup on /tmp or /scratch is not possible because another job could be still using data below these directories. Hence the corresponding file systems could fill up and this can cause issues for you and for other users. On the other hand, $TMPDIR is created when the job starts and removed when the job completes, i.e. a cleanup is automatically done.
			\|}

	== LSDF Online Storage==		== LSDF Online Storage==

R Laifer: /* $TMPDIR */

2024-10-29T09:44:54Z

$TMPDIR

← Older revision		Revision as of 11:44, 29 October 2024
Line 440:		Line 440:
	== $TMPDIR ==		== $TMPDIR ==

	The environment variable $TMPDIR contains the name of a directory which is local to each node. This means		The environment variable $TMPDIR contains the name of a directory which is located on the local SSD of each node. This means
	that different tasks of a parallel application use different directories when they do not utilize the same node.		that different tasks of a parallel application use different directories when they do not utilize the same node.
	Although $TMPDIR points to the same path name for different nodes of a batch job, the physical location and the		Although $TMPDIR points to the same path name for different nodes of a batch job, the physical location and the
Line 447:		Line 447:
	This directory should be used for temporary files being accessed from the local node during job runtime. It should		This directory should be used for temporary files being accessed from the local node during job runtime. It should
	also be used if you read the same data many times from a single node, e.g. if you are doing AI training. In this		also be used if you read the same data many times from a single node, e.g. if you are doing AI training. In this
	case you should copy the data at the beginning of your batch job to $TMPDIR and read the data from there.		case you should copy the data at the beginning of your batch job to $TMPDIR and read the data from there, see usage example below.

	The $TMPDIR directory is located on extremely fast ~~SSD~~ local storage devices. This means that performance		The $TMPDIR directory is located on extremely fast local SSD storage devices. This means that performance
	on small files is much better than on the parallel file systems. The capacity of the local SSDs for each node type		on small files is much better than on the parallel file systems. The capacity of the local SSDs for each node type
	is different and can be checked in Table 1 above. The capacity of $TMPDIR is at least 800 GB.		is different and can be checked in Table 1 above. The capacity of $TMPDIR is at least 800 GB.
Line 495:		Line 495:
	rsync -av $TMPDIR/results $(ws_find data-ssd)/results-${SLURM_JOB_ID}/		rsync -av $TMPDIR/results $(ws_find data-ssd)/results-${SLURM_JOB_ID}/
	</source>		</source>

			=== Use $TMPDIR instead of /tmp or /scratch ===

			Note that you should ''not'' use /tmp or /scratch and use $TMPDIR instead. The reason is that an automatic cleanup on /tmp or /scratch is not possible because another job could be still using data below these directories. Hence the corresponding file systems could fill up and this can cause issues for you and for other users. On the other hand, $TMPDIR is created when the job starts and removed when the job completes, i.e. a cleanup is automatically done.

	== LSDF Online Storage==		== LSDF Online Storage==

R Laifer: /* Linking workspaces in Home */

2024-10-17T17:50:30Z

Linking workspaces in Home

← Older revision		Revision as of 19:50, 17 October 2024
Line 325:		Line 325:
	=== Linking workspaces in Home ===		=== Linking workspaces in Home ===

	It might be valuable to have links to personal workspaces within a certain directory, e.g., below the user home directory. The command		It might be valuable to have links to personal workspaces within a certain directory, e.g. below the user home directory. The command
	<pre>		<pre>
	ws_register <DIR>		ws_register <DIR>

← Older revision		Revision as of 12:38, 17 January 2025
Line 44:		Line 44:
	! style="width:13%"\| Compute nodes "Thin"		! style="width:13%"\| Compute nodes "Thin"
	! style="width:13%"\| Compute nodes "HPC"		! style="width:13%"\| Compute nodes "HPC"
	! style="width:13%"\| Compute nodes "~~IceLake~~"		! style="width:13%"\| Compute nodes "Ice Lake"
	! style="width:13%"\| Compute nodes "Fat"		! style="width:13%"\| Compute nodes "Fat"
	! style="width:13%"\| GPU x4		! style="width:13%"\| GPU x4
	! style="width:13%"\| GPU x8		! style="width:13%"\| GPU x8
	! style="width:13%"\| ~~IceLake~~ + GPU x4		! style="width:13%"\| Ice Lake + GPU x4
	! style="width:13%"\| Login		! style="width:13%"\| Login
	\|-		\|-