BwForCluster JUSTUS 2 Slurm HOWTO - Revision history

J Salk at 17:35, 30 September 2025

2025-09-30T17:35:40Z

← Older revision		Revision as of 19:35, 30 September 2025
Line 409:		Line 409:
	# module load foo/bar		# module load foo/bar

	export OMP_NUM_THREADS=${~~SLURM_JOB_CPUS_PER_NODE~~}		export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}
	export MKL_NUM_THREADS=${~~SLURM_JOB_CPUS_PER_NODE~~}		export MKL_NUM_THREADS=${SLURM_CPUS_PER_TASK}

	# Run multithreaded program		# Run multithreaded program

J Salk: /* How to convert Moab batch job scripts to Slurm? */

2025-03-13T08:02:31Z

How to convert Moab batch job scripts to Slurm?

← Older revision		Revision as of 10:02, 13 March 2025
Line 147:		Line 147:
	\| Process count per node \|\| -l ppn=<count> \|\| --ntasks-per-node=<count>		\| Process count per node \|\| -l ppn=<count> \|\| --ntasks-per-node=<count>
	\|-		\|-
	\| Core count per process \|\| \|\| --cpus-per-task=<count>		\| Core count per process \|\| \|\| --cpus-per-task=<count> (-c <count>)
	\|-		\|-
	\| Memory limit per node \|\| -l mem=<limit> \|\| --mem=<limit>		\| Memory limit per node \|\| -l mem=<limit> \|\| --mem=<limit>

J Salk at 15:21, 23 October 2024

2024-10-23T15:21:00Z

← Older revision		Revision as of 17:21, 23 October 2024
Line 1,117:		Line 1,117:

	<pre>		<pre>
	$ sacctmgr modify user where user=<username> set DefaultAccount=<default_account>		$ sacctmgr modify user where user=<username> set DefaultAccount=<default_account>
	</pre>		</pre>

J Salk at 15:19, 23 October 2024

2024-10-23T15:19:01Z

← Older revision		Revision as of 17:19, 23 October 2024
Line 1,114:		Line 1,114:
	</pre>		</pre>

	== How to change default account of a user?		== How to change default account of a user? ==

	<pre>		<pre>

J Salk at 15:17, 23 October 2024

2024-10-23T15:17:28Z

← Older revision		Revision as of 17:17, 23 October 2024
Line 1,113:		Line 1,113:
	$ sacctmgr remove user <username> where account=<accountname> # Remove user from this account		$ sacctmgr remove user <username> where account=<accountname> # Remove user from this account
	</pre>		</pre>

			== How to change default account of a user?

			<pre>
			$ sacctmgr modify user where user=<username> set DefaultAccount=<default_account>
			</pre>

			'''Note:''' The user must already be associated with the account you want to set as default.

	== How to show account information? ==		== How to show account information? ==

J Salk: /* How to find working sample scripts for my program? */

2022-12-02T15:34:44Z

How to find working sample scripts for my program?

@@ Line 82: / Line 82: @@
 $ module help math/matlab   # display module help for Matlab
 </pre>
 == How to submit an interactive job? ==

J Salk: /* How to submit a multithreaded batch job? */

2022-09-20T23:34:07Z

How to submit a multithreaded batch job?

← Older revision		Revision as of 01:34, 21 September 2022
Line 408:		Line 408:

	* In our configuration each physical core is considered a "CPU".		* In our configuration each physical core is considered a "CPU".

			* On JUSTUS 2 it is recommended to specify a number of cores per task ('--cpus-per-task') that is either an integer divisor of 24 or (at most) 48.

	* Required memory can also by specified per allocated CPU with '--mem-per-cpu' option.		* Required memory can also by specified per allocated CPU with '--mem-per-cpu' option.

	* The '--mem' and '--mem-per-cpu' options are mutually exclusive.		* The '--mem' and '--mem-per-cpu' options are mutually exclusive.

	* In terms of core allocation '--ntasks-per-node=1' or '--ntasks=1' together with '--cpus-per-task=8' is almost equivalent to '--ntasks-per-node=8' or '--ntasks=8' and omitting '--cpus-per-task=8'. However, there are subtle differences when multiple tasks are spawned within one job by means of srun command.
	** See: https://stackoverflow.com/questions/39186698/what-does-the-ntasks-or-n-tasks-does-in-slurm

	== How to submit an array job? ==		== How to submit an array job? ==

J Salk: /* How to retrieve historical resource usage records for a specific user or account? */

2022-08-15T15:37:24Z

How to retrieve historical resource usage records for a specific user or account?

← Older revision		Revision as of 17:37, 15 August 2022
Line 1,138:		Line 1,138:
	</pre>		</pre>

	== How to retrieve historical resource usage ~~records~~ for a specific user or account? ==		== How to retrieve historical resource usage for a specific user or account? ==

	Use [https://slurm.schedmd.com/sreport.html sreport] command.		Use [https://slurm.schedmd.com/sreport.html sreport] command.

J Salk: /* How to retrieve historical resource usage records for a specific user? */

2022-08-15T15:33:13Z

How to retrieve historical resource usage records for a specific user?

← Older revision		Revision as of 17:33, 15 August 2022
Line 1,138:		Line 1,138:
	</pre>		</pre>

	== How to retrieve historical resource usage records for a specific user? ==		== How to retrieve historical resource usage records for a specific user or account? ==

	Use [https://slurm.schedmd.com/sreport.html sreport] command.		Use [https://slurm.schedmd.com/sreport.html sreport] command.

J Salk: /* How to retrieve historical resource usage records for a specific user? */

2022-07-26T21:45:07Z

How to retrieve historical resource usage records for a specific user?

← Older revision		Revision as of 23:45, 26 July 2022
Line 1,142:		Line 1,142:
	Use [https://slurm.schedmd.com/sreport.html sreport] command.		Use [https://slurm.schedmd.com/sreport.html sreport] command.

			Examples:
	Example:
	<pre>		<pre>
	$ sreport cluster UserUtilizationByAccount Start=2021-01-01 End=2021-12-31 -t Hours user=<username>		$ sreport cluster UserUtilizationByAccount Start=2021-01-01 End=2021-12-31 -t Hours user=<username> # Report cluster utilization of given user broken down by accounts
			$ sreport cluster AccountUtilizationByUser Start=2021-01-01 End=2021-12-31 -t Hours account=<account> # Report cluster utilization of given account broken down by users
	</pre>		</pre>

	'''Notes:'''		'''Notes:'''
	* By default CPU resources will be reported. Use '-T' option for other trackable resources, e.g. '-T cpu,mem,gres/gpu,gres/scratch'.		* By default CPU resources will be reported. Use '-T' option for other trackable resources, e.g. '-T cpu,mem,gres/gpu,gres/scratch'.
			* On JUSTUS 2 registered compute projects ("Rechenvorhaben") are uniquely mapped to Slurm accounts of the same name. Thus, 'AccountUtilizationByUser' can also be used to report the aggregated cluster utilization of compute projects.
	* Can be executed by regular users as well in which case Slurm will ~~always~~ report their own usage records ~~only~~.		* Can be executed by regular users as well in which case Slurm will only report their own usage records (but along with the total usage of the associated account in the case of 'AccountUtilizationByUser').

	== How to fix/reset a user's RawUsage value? ==		== How to fix/reset a user's RawUsage value? ==