NEMO2/Hardware - Revision history

M Janczyk: /* Listing File System Quotas */

2026-02-24T10:24:12Z

Listing File System Quotas

← Older revision		Revision as of 12:24, 24 February 2026
Line 236:		Line 236:
	=== Listing File System Quotas ===		=== Listing File System Quotas ===

	You can list the quotas for your ~~users~~ using <code>nemoquota</code> (prints <code>df --si</code>) or <code>df <path_to_directory></code>.		You can list the quotas for your user using <code>nemoquota</code> (prints <code>df --si</code>) or <code>df <path_to_directory></code>.

	Examples:		Examples:

M Janczyk: /* Compute and Special Purpose Nodes */

2026-02-02T16:32:27Z

Compute and Special Purpose Nodes

M Janczyk: rtx node

2026-02-02T16:25:45Z

rtx node

← Older revision		Revision as of 18:25, 2 February 2026
Line 39:		Line 39:
	! style="width:15%"\| L40S Partition		! style="width:15%"\| L40S Partition
	! style="width:15%"\| MI300A Partition		! style="width:15%"\| MI300A Partition
	! style="width:15%"\| H200 Partition ~~(August 2025)~~		! style="width:15%"\| H200 Partition
			! style="width:15%"\| RTX Partition
	! style="width:15%"\| Login Partition		! style="width:15%"\| Login Partition
	\|-		\|-
Line 48:		Line 49:
	\| 4		\| 4
	\| 2		\| 2
			\| 1
	\| 2		\| 2
	\|-		\|-
Line 58:		Line 60:
	\| 2x [https://www.amd.com/en/products/processors/server/epyc/4th-generation-9004-and-8004-series/amd-epyc-9654.html AMD EPYC 9654 (Genoa)]		\| 2x [https://www.amd.com/en/products/processors/server/epyc/4th-generation-9004-and-8004-series/amd-epyc-9654.html AMD EPYC 9654 (Genoa)]
	8x [https://www.nvidia.com/en-us/data-center/h200/ NVIDIA H200]		8x [https://www.nvidia.com/en-us/data-center/h200/ NVIDIA H200]
			\| 2x [https://www.amd.com/en/products/processors/server/epyc/9005-series/amd-epyc-9655.html AMD EPYC 9655 (Turin)]
			4x [https://www.nvidia.com/en-us/data-center/rtx-pro-6000-blackwell-server-edition/ NVIDIA RTX PRO 6000 Blackwell Server Edition]
	\| 1x [https://www.amd.com/en/products/processors/server/epyc/4th-generation-9004-and-8004-series/amd-epyc-9354.html AMD EPYC 9354 (Genoa)]		\| 1x [https://www.amd.com/en/products/processors/server/epyc/4th-generation-9004-and-8004-series/amd-epyc-9354.html AMD EPYC 9354 (Genoa)]
	\|-		\|-
Line 69:		Line 73:
	\| 2.4/3.55		\| 2.4/3.55
	34 (FP64) / 67 (FP32) / 3958 (INT8)		34 (FP64) / 67 (FP32) / 3958 (INT8)
			\| 2.6/4.1
			120 (FP32) / 4000 (FP4)
	\| 3.25/3.75		\| 3.25/3.75
	\|-		\|-
Line 80:		Line 86:
	\| 4x 24		\| 4x 24
	'''Usable in Slurm: 92'''		'''Usable in Slurm: 92'''
			\| 192
			'''Usable in Slurm: 190'''
	\| 192		\| 192
	'''Usable in Slurm: 190'''		'''Usable in Slurm: 190'''
Line 91:		Line 99:
	\| 23		\| 23
	\| 23		\| 23
			\| 47
	\| ---		\| ---
	\|-		\|-
Line 105:		Line 114:
	\| 1536 GiB, 4.8 GHz (DDR5)		\| 1536 GiB, 4.8 GHz (DDR5)
	8x 141 GB, 4813 GB/s (HBM3e)		8x 141 GB, 4813 GB/s (HBM3e)
			'''Usable in Slurm: 1459 GiB, 1495000 MiB, per Core:7800 MiB'''
			\| 1536 GiB, 6 GHz (DDR5)
			4x 96 GB, 1597 GB/s (GDDR7)
	'''Usable in Slurm: 1459 GiB, 1495000 MiB, per Core:7800 MiB'''		'''Usable in Slurm: 1459 GiB, 1495000 MiB, per Core:7800 MiB'''
	\| 384 GiB, 4.8 GHz (DDR5)		\| 384 GiB, 4.8 GHz (DDR5)
Line 112:		Line 124:
	\| 3840		\| 3840
	\| 1920		\| 1920
			\| 3840
	\| 3840		\| 3840
	\| 3840		\| 3840
Line 121:		Line 134:
	\| Omni-Path 100		\| Omni-Path 100
	100 GbE (RoCEv2)		100 GbE (RoCEv2)
			\| 100 GbE (RoCEv2)
	\| 100 GbE (RoCEv2)		\| 100 GbE (RoCEv2)
	\| 100 GbE (RoCEv2)		\| 100 GbE (RoCEv2)
Line 145:		Line 159:
	\|colspan="6" \| Maximum resources for a single node job (*):		\|colspan="6" \| Maximum resources for a single node job (*):
	<code>--partition=h200 --ntasks=190 --gres=gpu:8 --mem=1459GB # or: --mem=1495000MB, or: --mem-per-cpu=7800MB</code>		<code>--partition=h200 --ntasks=190 --gres=gpu:8 --mem=1459GB # or: --mem=1495000MB, or: --mem-per-cpu=7800MB</code>
			\|-
			!scope="column" \| Job Example RTX Partition
			\|colspan="6" \| Maximum resources for a single node job (*):
			<code>--partition=rtx --ntasks=190 --gres=gpu:4 --mem=727GB # or: --mem=745000MB, or: --mem-per-cpu=3900MB</code>
	\|-		\|-
	!scope="column" \| Job Example Login Partition		!scope="column" \| Job Example Login Partition

M Janczyk: /* File Systems */

2025-11-17T16:28:49Z

File Systems

@@ Line 204: / Line 204: @@
   batch job walltime : files are removed at end of the batch job
 </pre>
 === Listing File System Quotas ===

M Janczyk: /* File Systems */

2025-11-17T16:24:25Z

File Systems

← Older revision		Revision as of 18:24, 17 November 2025
Line 187:		Line 187:
	\|-		\|-
	!scope="column" \| Snapshots		!scope="column" \| Snapshots
			\| hourly / daily / weekly [https://www.nemo.uni-freiburg.de/news/2025/11/17/snaphots-home/]
	\| daily (7 snapshots) (not yet implemented)
	\| ---		\| ---
	\| ---		\| ---

M Janczyk: /* Listing File System Quotas */

2025-10-06T14:48:01Z

Listing File System Quotas

← Older revision		Revision as of 16:48, 6 October 2025
Line 207:		Line 207:
	=== Listing File System Quotas ===		=== Listing File System Quotas ===

	You can list the quotas for your users using <code>nemoquota</code> or <code>df <path_to_directory></code>.		You can list the quotas for your users using <code>nemoquota</code> (prints <code>df --si</code>) or <code>df <path_to_directory></code>.

	Examples:		Examples:

M Janczyk: /* File Systems */

2025-10-06T14:46:22Z

File Systems

← Older revision		Revision as of 16:46, 6 October 2025
Line 164:		Line 164:
	! style="width:10%"\|		! style="width:10%"\|
	! style="width:10%"\| <tt>$HOME</tt>		! style="width:10%"\| <tt>$HOME</tt>
	! style="width:10%"\| ~~'''~~[[Workspace]]s~~'''~~		! style="width:10%"\| [[Workspace]]s
	! style="width:10%"\| NVMe		! style="width:10%"\| NVMe
	\|-		\|-

M Janczyk at 14:46, 6 October 2025

2025-10-06T14:46:03Z

← Older revision		Revision as of 16:46, 6 October 2025
Line 156:		Line 156:

	NEMO2 offers a fast [https://docs.weka.io/ Weka] parallel filesystem, which is only limited by the uplink to this storage (>90GB/s).		NEMO2 offers a fast [https://docs.weka.io/ Weka] parallel filesystem, which is only limited by the uplink to this storage (>90GB/s).
	The storage is used for <tt>$HOME</tt> and ~~workspaces~~.		The storage is used for <tt>$HOME</tt> and '''[[workspace]]s'''.
	There sill be no backups, but we plan to implement Snapshots for the last 7 days in the next months.		There sill be no backups, but we plan to implement Snapshots for the last 7 days in the next months.
	Additionally, each compute node provides temporary storage on the node-local NVMe disk.		Additionally, each compute node provides temporary storage on the node-local NVMe disk.
Line 164:		Line 164:
	! style="width:10%"\|		! style="width:10%"\|
	! style="width:10%"\| <tt>$HOME</tt>		! style="width:10%"\| <tt>$HOME</tt>
	! style="width:10%"\| ~~Workspaces~~		! style="width:10%"\| '''[[Workspace]]s'''
	! style="width:10%"\| NVMe		! style="width:10%"\| NVMe
	\|-		\|-

M Janczyk: /* Compute and Special Purpose Nodes */

2025-09-15T15:33:32Z

Compute and Special Purpose Nodes

← Older revision		Revision as of 17:33, 15 September 2025
Line 105:		Line 105:
	\| 1536 GiB, 4.8 GHz (DDR5)		\| 1536 GiB, 4.8 GHz (DDR5)
	8x 141 GB, 4813 GB/s (HBM3e)		8x 141 GB, 4813 GB/s (HBM3e)
	'''Usable in Slurm: GiB, MiB, per Core: MiB'''		'''Usable in Slurm: 1459 GiB, 1495000 MiB, per Core:7800 MiB'''
	\| 384 GiB, 4.8 GHz (DDR5)		\| 384 GiB, 4.8 GHz (DDR5)
	'''Usable for Users: 50 GiB, 51200 MiB, per Core: 12800 MiB'''		'''Usable for Users: 50 GiB, 51200 MiB, per Core: 12800 MiB'''

M Janczyk at 15:25, 15 September 2025

2025-09-15T15:25:07Z

← Older revision		Revision as of 17:25, 15 September 2025
Line 144:		Line 144:
	!scope="column" \| Job Example H200 Partition		!scope="column" \| Job Example H200 Partition
	\|colspan="6" \| Maximum resources for a single node job (*):		\|colspan="6" \| Maximum resources for a single node job (*):
	<code>--partition=h200 --ntasks=190 --gres=gpu:8 --mem=GB # or: --mem=MB, or: --mem-per-cpu=MB</code>		<code>--partition=h200 --ntasks=190 --gres=gpu:8 --mem=1459GB # or: --mem=1495000MB, or: --mem-per-cpu=7800MB</code>
	\|-		\|-
	!scope="column" \| Job Example Login Partition		!scope="column" \| Job Example Login Partition
	\|colspan="6" \| Maximum resources for a single node job (*):		\|colspan="6" \| Maximum resources for a single node job (*):
	<code>--partition=login --ntasks=4 --time=15:00 --mem=50GB + or: --mem=~~51201MB~~, or: --mem-per-cpu=12800MB</code>		<code>--partition=login --ntasks=4 --time=15:00 --mem=50GB + or: --mem=51200MB, or: --mem-per-cpu=12800MB</code>
	\|-		\|-
	\|}		\|}
Line 179:		Line 179:
	!scope="column" \| Capacity		!scope="column" \| Capacity
	\|colspan="2" style="text-align:center;"\| 1 PB		\|colspan="2" style="text-align:center;"\| 1 PB
	\| 1.9 TB ~~or more~~ (~~depends~~ on ~~node~~)		\| 1.92/3.84 TB (logins 480 GB)
	\|-		\|-
	!scope="column" \| Quotas per <tt>$HOME</tt>/Workspace		!scope="column" \| Quotas per <tt>$HOME</tt>/Workspace

← Older revision		Revision as of 16:46, 6 October 2025
Line 164:		Line 164:
	! style="width:10%"\|		! style="width:10%"\|
	! style="width:10%"\| <tt>$HOME</tt>		! style="width:10%"\| <tt>$HOME</tt>
	! style="width:10%"\| ~~'''~~[[Workspace]]s~~'''~~		! style="width:10%"\| [[Workspace]]s
	! style="width:10%"\| NVMe		! style="width:10%"\| NVMe
	\|-		\|-

← Older revision		Revision as of 18:32, 2 February 2026
Line 34:		Line 34:
	{\| class="wikitable" style="text-align:center;"		{\| class="wikitable" style="text-align:center;"
	\|-		\|-
	! style="width:15%"\|		! style="width:14%"\|
	! style="width:15%"\| Genoa Partition		! style="width:10%"\| Genoa Partition
	! style="width:15%"\| Milan Partition		! style="width:10%"\| Milan Partition
	! style="width:15%"\| L40S Partition		! style="width:14%"\| L40S Partition
	! style="width:15%"\| MI300A Partition		! style="width:12%"\| MI300A Partition
	! style="width:15%"\| H200 Partition		! style="width:14%"\| H200 Partition
	! style="width:15%"\| RTX Partition		! style="width:14%"\| RTX Partition
	! style="width:15%"\| Login Partition		! style="width:12%"\| Login Partition
	\|-		\|-
	!scope="column"\| Quantity		!scope="column"\| Quantity
Line 141:		Line 141:
	\|-		\|-
	!scope="column" \| Job Example Genoa Partition		!scope="column" \| Job Example Genoa Partition
	\|colspan="6" \| Maximum resources for a single node job (*):		\|colspan="7" \| Maximum resources for a single node job (*):
	<code>--partition=genoa --ntasks=190 --mem=727GB # or: --mem=745000MB, or: --mem-per-cpu=3900MB</code>		<code>--partition=genoa --ntasks=190 --mem=727GB # or: --mem=745000MB, or: --mem-per-cpu=3900MB</code>
	\|-		\|-
	!scope="column" \| Job Example Milan Partition		!scope="column" \| Job Example Milan Partition
	\|colspan="6" \| Maximum resources for a single node job (*):		\|colspan="7" \| Maximum resources for a single node job (*):
	<code>--partition=milan --ntasks=126 --mem=495GB # or: --mem=507000MB, or: --mem-per-cpu=4000MB</code>		<code>--partition=milan --ntasks=126 --mem=495GB # or: --mem=507000MB, or: --mem-per-cpu=4000MB</code>
	\|-		\|-
	!scope="column" \| Job Example L40S Partition		!scope="column" \| Job Example L40S Partition
	\|colspan="6" \| Maximum resources for a single node job (*):		\|colspan="7" \| Maximum resources for a single node job (*):
	<code>--partition=l40s --ntasks=62 --gres=gpu:4 --mem=495GB # or: --mem=507000MB, or: --mem-per-cpu=8100MB</code>		<code>--partition=l40s --ntasks=62 --gres=gpu:4 --mem=495GB # or: --mem=507000MB, or: --mem-per-cpu=8100MB</code>
	\|-		\|-
	!scope="column" \| Job Example MI300A Partition		!scope="column" \| Job Example MI300A Partition
	\|colspan="6" \| Maximum resources for a single node job (*):		\|colspan="7" \| Maximum resources for a single node job (*):
	<code>--partition=mi300a --ntasks=94 --gres=gpu:4 --mem=495GB # or: --mem=507000MB, or: --mem-per-cpu=5300MB</code>		<code>--partition=mi300a --ntasks=94 --gres=gpu:4 --mem=495GB # or: --mem=507000MB, or: --mem-per-cpu=5300MB</code>
	\|-		\|-
	!scope="column" \| Job Example H200 Partition		!scope="column" \| Job Example H200 Partition
	\|colspan="6" \| Maximum resources for a single node job (*):		\|colspan="7" \| Maximum resources for a single node job (*):
	<code>--partition=h200 --ntasks=190 --gres=gpu:8 --mem=1459GB # or: --mem=1495000MB, or: --mem-per-cpu=7800MB</code>		<code>--partition=h200 --ntasks=190 --gres=gpu:8 --mem=1459GB # or: --mem=1495000MB, or: --mem-per-cpu=7800MB</code>
	\|-		\|-
	!scope="column" \| Job Example RTX Partition		!scope="column" \| Job Example RTX Partition
	\|colspan="6" \| Maximum resources for a single node job (*):		\|colspan="7" \| Maximum resources for a single node job (*):
	<code>--partition=rtx --ntasks=190 --gres=gpu:4 --mem=727GB # or: --mem=745000MB, or: --mem-per-cpu=3900MB</code>		<code>--partition=rtx --ntasks=190 --gres=gpu:4 --mem=727GB # or: --mem=745000MB, or: --mem-per-cpu=3900MB</code>
	\|-		\|-
	!scope="column" \| Job Example Login Partition		!scope="column" \| Job Example Login Partition
	\|colspan="6" \| Maximum resources for a single node job (*):		\|colspan="7" \| Maximum resources for a single node job (*):
	<code>--partition=login --ntasks=4 --time=15:00 --mem=50GB + or: --mem=51200MB, or: --mem-per-cpu=12800MB</code>		<code>--partition=login --ntasks=4 --time=15:00 --mem=50GB + or: --mem=51200MB, or: --mem-per-cpu=12800MB</code>
	\|-		\|-