JUSTUS3/Getting Started - Revision history

K Siegmund at 14:46, 24 June 2026

2026-06-24T14:46:22Z

← Older revision		Revision as of 16:46, 24 June 2026
Line 30:		Line 30:
	<!-- TODO: review and put script on own page and link here [[BwForCluster_JUSTUS_2_Slurm_HOWTO#How_to_clean-up_or_save_files_before_a_job_times_out.3F\| Slurm Howto]] shows how to copy and clean up your data from /tmp or /scratch at the end of the job -->		<!-- TODO: review and put script on own page and link here [[BwForCluster_JUSTUS_2_Slurm_HOWTO#How_to_clean-up_or_save_files_before_a_job_times_out.3F\| Slurm Howto]] shows how to copy and clean up your data from /tmp or /scratch at the end of the job -->
	* Submit and monitor your jobs with Slurm commands.		* Submit and monitor your jobs with Slurm commands.
	** → '''[[~~JUSTUS2~~/Jobscripts: Running Your Calculations\|Jobscripts: Running Your Calculations]]'''		** → '''[[JUSTUS3/Jobscripts: Running Your Calculations\|Jobscripts: Running Your Calculations]]'''
	** → '''[[BwForCluster_JUSTUS_2_Slurm_HOWTO\| extensive Slurm HOWTO on specific tasks]]'''		** → '''[[BwForCluster_JUSTUS_2_Slurm_HOWTO\| extensive Slurm HOWTO on specific tasks]]'''

Line 39:		Line 39:
	* Learn about Scaling your Job		* Learn about Scaling your Job
	** Determine how many compute-cores the job can use? → '''[[Scaling]]'''		** Determine how many compute-cores the job can use? → '''[[Scaling]]'''
	** Make especially sure your compute jobs are running efficiently &~~arr~~; '''[[Energy Efficient Cluster Usage]]'''		** Make especially sure your compute jobs are running efficiently → '''[[Energy Efficient Cluster Usage]]'''

	* Specially remember to mention the cluster in your publications. → '''[[bwForCluster JUSTUS 3 Acknowledgement\|Acknowledgement]]'''		* Specially remember to mention the cluster in your publications. → '''[[bwForCluster JUSTUS 3 Acknowledgement\|Acknowledgement]]'''

K Siegmund at 14:45, 24 June 2026

2026-06-24T14:45:28Z

← Older revision		Revision as of 16:45, 24 June 2026
Line 7:		Line 7:

	== Basics ==		== Basics ==
	* know the cluster rules: → '''[[~~JUSTUS2~~/Policy_Agreement]]'''		* know the cluster rules: → '''[[JUSTUS3/Policy_Agreement]]'''
	* log in to the cluster (and first set a 2FA secret): '''[[JUSTUS3/Login]]'''		* log in to the cluster (and first set a 2FA secret): '''[[JUSTUS3/Login]]'''
	* get accustomed with the linux commandline:		* get accustomed with the linux commandline:
Line 25:		Line 25:
	* transfer your own data to the cluster → '''[[Data Transfer]]'''		* transfer your own data to the cluster → '''[[Data Transfer]]'''
	** Learn about available Hardware:		** Learn about available Hardware:
	*** → '''[[~~Hardware_and_Architecture_(bwForCluster_JUSTUS_2)#Storage_Architecture\|~~File Systems]]'''		*** → '''[[JUSTUS3/File Systems]]'''
	*** → '''[[~~Hardware_and_Architecture_(bwForCluster_JUSTUS_2)\|Hardware~~ ~~and~~ ~~Architecture~~]]'''		*** → '''[[JUSTUS3/Compute Node Types]]'''
	* adapt the sample job script to run your own job		* adapt the sample job script to run your own job
	<!-- TODO: review and put script on own page and link here [[BwForCluster_JUSTUS_2_Slurm_HOWTO#How_to_clean-up_or_save_files_before_a_job_times_out.3F\| Slurm Howto]] shows how to copy and clean up your data from /tmp or /scratch at the end of the job -->		<!-- TODO: review and put script on own page and link here [[BwForCluster_JUSTUS_2_Slurm_HOWTO#How_to_clean-up_or_save_files_before_a_job_times_out.3F\| Slurm Howto]] shows how to copy and clean up your data from /tmp or /scratch at the end of the job -->

K Siegmund: /* Running an Example with Preinstalled Software */

2026-06-24T14:36:38Z

Running an Example with Preinstalled Software

← Older revision		Revision as of 16:36, 24 June 2026
Line 19:		Line 19:
	* continue reading until you found that there are example job scripts: '''[[Environment_Modules#Software_job_examples]]'''		* continue reading until you found that there are example job scripts: '''[[Environment_Modules#Software_job_examples]]'''
	* submit a sample job from a software as mentioned in the job example. Also see: '''[[JUSTUS3/Slurm]]'''		* submit a sample job from a software as mentioned in the job example. Also see: '''[[JUSTUS3/Slurm]]'''
	* monitor your job: '''[[JUSTUS3/~~Monitoring_Your_Jobs~~]]'''		* monitor your job: '''[[JUSTUS3/Monitoring Your Jobs]]'''

	== Running Your Own Calculations in a Batch Job ==		== Running Your Own Calculations in a Batch Job ==

K Siegmund: /* Basics */

2026-06-24T14:36:24Z

Basics

← Older revision		Revision as of 16:36, 24 June 2026
Line 8:		Line 8:
	== Basics ==		== Basics ==
	* know the cluster rules: → '''[[JUSTUS2/Policy_Agreement]]'''		* know the cluster rules: → '''[[JUSTUS2/Policy_Agreement]]'''
	* log in to the cluster (and first set a 2FA secret): [[JUSTUS3/Login]]		* log in to the cluster (and first set a 2FA secret): '''[[JUSTUS3/Login]]'''
	* get accustomed with the linux commandline:		* get accustomed with the linux commandline:
	** HPC Wiki (external site) → '''[https://hpc-wiki.info/hpc/Introduction_to_Linux_in_HPC/The_Command_Line Introduction to Linux Commandline]'''		** HPC Wiki (external site) → '''[https://hpc-wiki.info/hpc/Introduction_to_Linux_in_HPC/The_Command_Line Introduction to Linux Commandline]'''

K Siegmund: Created page with " Here is a short list of things you may need to do first when you get onto the cluster == Pre-Requisites == * Know how a HPC compute Cluster works: In short: you do not simply run log in and start your software, but you write a "job script" that contains all commands to run and process your job and send this into a waiting queue to be run on one of several hundred computers. → '''Running Calculations''' * Get Access: Follow the registration process for the bw..."

2026-06-24T14:36:03Z

Created page with " Here is a short list of things you may need to do first when you get onto the cluster == Pre-Requisites == * Know how a HPC compute Cluster works: In short: you do not simply run log in and start your software, but you write a "job script" that contains all commands to run and process your job and send this into a waiting queue to be run on one of several hundred computers. → '''Running Calculations''' * Get Access: Follow the registration process for the bw..."

New page

Here is a short list of things you may need to do first when you get onto the cluster

== Pre-Requisites ==
* Know how a HPC compute Cluster works: In short: you do not simply run log in and start your software, but you write a "job script" that contains all commands to run and process your job and send this into a waiting queue to be run on one of several hundred computers. → '''[[Running Calculations]]'''
* Get Access: Follow the registration process for the bwForCluster. → [[Registration/bwForCluster|How to Register for a bwForCluster]]

== Basics ==
* know the cluster rules: → '''[[JUSTUS2/Policy_Agreement]]'''
* log in to the cluster (and first set a 2FA secret): [[JUSTUS3/Login]]
* get accustomed with the linux commandline:
** HPC Wiki (external site) → '''[https://hpc-wiki.info/hpc/Introduction_to_Linux_in_HPC/The_Command_Line Introduction to Linux Commandline]'''
** Have commands ready → '''[[Media:HPC_User_Linux_command_cheatsheet.pdf|HPC User's Linux Cheat Sheet (PDF)]]'''
** configure your shell → '''[[.bashrc Do's and Don'ts]]'''
** linux course at '''[https://training.bwhpc.de/ training.bwhpc.de]'''

== Running an Example with Preinstalled Software ==
* scientific software: learn how to load '''[[Software Modules]]'''
* continue reading until you found that there are example job scripts: '''[[Environment_Modules#Software_job_examples]]'''
* submit a sample job from a software as mentioned in the job example. Also see: '''[[JUSTUS3/Slurm]]'''
* monitor your job: '''[[JUSTUS3/Monitoring_Your_Jobs]]'''

== Running Your Own Calculations in a Batch Job ==
{{Template:AttentionBox|Compute jobs on nodes '''must not''' write temporary I/O intensive data to central file systems.}}
* transfer your own data to the cluster → '''[[Data Transfer]]'''
** Learn about available Hardware:
*** → '''[[Hardware_and_Architecture_(bwForCluster_JUSTUS_2)#Storage_Architecture|File Systems]]'''
*** → '''[[Hardware_and_Architecture_(bwForCluster_JUSTUS_2)|Hardware and Architecture]]'''
* adapt the sample job script to run your own job

* Submit and monitor your jobs with Slurm commands.
** → '''[[JUSTUS2/Jobscripts: Running Your Calculations|Jobscripts: Running Your Calculations]]'''
** → '''[[BwForCluster_JUSTUS_2_Slurm_HOWTO| extensive Slurm HOWTO on specific tasks]]'''

* Remember to mention the cluster in your publications. → '''[[bwForCluster JUSTUS 3 Acknowledgement|Acknowledgement]]'''

== Do Production Runs with Hundreds of Jobs ==

* Learn about Scaling your Job
** Determine how many compute-cores the job can use? → '''[[Scaling]]'''
** Make especially sure your compute jobs are running efficiently &arr; '''[[Energy Efficient Cluster Usage]]'''

* Specially remember to mention the cluster in your publications. → '''[[bwForCluster JUSTUS 3 Acknowledgement|Acknowledgement]]'''