When to use an HPC Cluster - Revision history

S Fischer: feedback from bwCloud colleagues

2026-09-16T08:16:38Z

feedback from bwCloud colleagues

← Older revision		Revision as of 10:16, 16 September 2026
Line 31:		Line 31:
	* having an environment for student projects or theses,		* having an environment for student projects or theses,

	then the bwCloud offering virtual machines might be the right thing for you		then the bwCloud-OS offering virtual machines might be the right thing for you

	'''→ https://bwcloud-os.de/'''		'''→ https://bwcloud-os.de/'''

K Siegmund: /* When not to use HPC computers */

2026-09-14T11:32:21Z

When not to use HPC computers

← Older revision		Revision as of 13:32, 14 September 2026
Line 33:		Line 33:
	then the bwCloud offering virtual machines might be the right thing for you		then the bwCloud offering virtual machines might be the right thing for you

	~~```~~→ https://bwcloud-os.de/~~```~~		'''→ https://bwcloud-os.de/'''

	== Misconceptions ==		== Misconceptions ==

K Siegmund: /* When not to use HPC computers */

2026-09-14T11:31:58Z

When not to use HPC computers

← Older revision		Revision as of 13:31, 14 September 2026
Line 19:		Line 19:
	* Your calculation runs in serial (only on one compute core is used, things cannot run in parallel) and you have few calculations		* Your calculation runs in serial (only on one compute core is used, things cannot run in parallel) and you have few calculations
	* Your workflow requires that you run one calculation, then retrieve data to analyze it locally, then run the next calculation		* Your workflow requires that you run one calculation, then retrieve data to analyze it locally, then run the next calculation
			* You have different computational needs than raw computing power.

	Except 3) or 4) of "when to use a cluster" force you to use the clusters anyway.		Except 3) or 4) of "when to use a cluster" force you to use the clusters anyway.

			If you need other computational tasks of the likes of

			* hosting a web server
			* hosting a database
			* having a test bed for software development
			* having systems for university courses on linux, programming languages, etc
			* having an environment for student projects or theses,

			then the bwCloud offering virtual machines might be the right thing for you

			```→ https://bwcloud-os.de/```

	== Misconceptions ==		== Misconceptions ==

S Fischer: change link to English website

2026-08-25T08:38:49Z

change link to English website

← Older revision		Revision as of 10:38, 25 August 2026
Line 38:		Line 38:
	=== "I want to host servers on bwHPC" ===		=== "I want to host servers on bwHPC" ===

	On the bwHPC clusters, you submit jobs to a supercomputer that get scheduled and processed over time. It is not the place for permanent server hosting. If you need self-service virtual machines, check out [https://bwcloud-os.de/ bwCloud-OS], which offers an OpenStack-based [https://en.wikipedia.org/wiki/Infrastructure_as_a_service IaaS] platform for research, teaching, and administration in Baden-Württemberg.		On the bwHPC clusters, you submit jobs to a supercomputer that get scheduled and processed over time. It is not the place for permanent server hosting. If you need self-service virtual machines, check out [https://bwcloud-os.de/en/ bwCloud-OS], which offers an OpenStack-based [https://en.wikipedia.org/wiki/Infrastructure_as_a_service IaaS] platform for research, teaching, and administration in Baden-Württemberg.

	== What are the Costs? ==		== What are the Costs? ==

S Fischer: more details on bwCloud-OS

2026-08-25T08:37:16Z

more details on bwCloud-OS

← Older revision		Revision as of 10:37, 25 August 2026
Line 38:		Line 38:
	=== "I want to host servers on bwHPC" ===		=== "I want to host servers on bwHPC" ===

	On the bwHPC-clusters, you submit jobs to a supercomputer that get scheduled and processed over time. It is not the place for permanent server hosting. If you need self-service virtual machines, check out ~~the~~ [https://bwcloud-os.de/ bwCloud-OS].		On the bwHPC clusters, you submit jobs to a supercomputer that get scheduled and processed over time. It is not the place for permanent server hosting. If you need self-service virtual machines, check out [https://bwcloud-os.de/ bwCloud-OS], which offers an OpenStack-based [https://en.wikipedia.org/wiki/Infrastructure_as_a_service IaaS] platform for research, teaching, and administration in Baden-Württemberg.

	== What are the Costs? ==		== What are the Costs? ==

S Fischer: updated link

2026-08-25T08:30:17Z

updated link

← Older revision		Revision as of 10:30, 25 August 2026
Line 38:		Line 38:
	=== "I want to host servers on bwHPC" ===		=== "I want to host servers on bwHPC" ===

	On the bwHPC-clusters, you submit jobs to a supercomputer that get scheduled and processed over time. It is not the place for permanent server hosting. If you need self-service virtual machines, check out the [https://~~www.bw~~-~~cloud~~.~~org~~/ bwCloud].		On the bwHPC-clusters, you submit jobs to a supercomputer that get scheduled and processed over time. It is not the place for permanent server hosting. If you need self-service virtual machines, check out the [https://bwcloud-os.de/ bwCloud-OS].

	== What are the Costs? ==		== What are the Costs? ==

H Schumacher: fixed typo in title

2026-04-13T08:27:12Z

fixed typo in title

New page

I have calculations to do - should I try using one of the bwHPC clusters for my tasks?

== When to use HPC computers ==

You can benefit from an HPC-Cluster if:

# You can run many separate calculations in parallel
# Software can efficiently divide your problem into many smaller problems that run in parallel
# You need more RAM (memory) than your computer can provide
# You need more GPUs and have no other source
# You need to process large amounts of data that do not fit on your computer

== When '''not''' to use HPC computers ==

You will probably not have a big benefit from using HPC when:

* Your calculation runs in serial (only on one compute core is used, things cannot run in parallel) and you have few calculations
* Your workflow requires that you run one calculation, then retrieve data to analyze it locally, then run the next calculation

Except 3) or 4) of "when to use a cluster" force you to use the clusters anyway.

== Misconceptions ==
=== "Everything will be faster" ===
[[File:BwUniCluster_2.0_Feb2020_1024x423.jpg|right|thumb|alt=bwUniCluster2.0 |upright=1| many compute cores: Every "tray" in these racks contains 1-4 computers, every computer typically has 50-200 compute cores. ]]

The performance of a single compute core is not higher than that of your typical desktop computer. The main things we can offer are:

* Many compute cores
* Much RAM (memory)
* GPUs

=== "It's just a computer, I know how to use a computer" ===

While the clusters are indeed "just" many Linux computers, they are also multi-million Euro instruments. So while they may have many similarities with a Linux desktop, in some respects, it is better to think of them like of other expensive instruments you may use in your research. HPC clusters have hardware components you may have never heard of in the consumer market, they typically have more than one CPU socket (making among other things memory management more complicated), etc. etc. etc.

=== "I want to host servers on bwHPC" ===

On the bwHPC-clusters, you submit jobs to a supercomputer that get scheduled and processed over time. It is not the place for permanent server hosting. If you need self-service virtual machines, check out the [https://www.bw-cloud.org/ bwCloud].

== What are the Costs? ==

=== Monetary ===

No cost to you as the end-user

(But of course the HPC systems are bought for millions of Euro and the power cost of running calculations is in the same order of magnitude as buying the systems)

=== Effort / Time ===

There is quite a learning curve to start calculating.

* Linux shell knowledge is a major part
* HPC-specific knowledge:
** Software module system - software supplied by the cluster maintainers (vide infra)
** How to use the scheduler / write job scripts: how to send calculations to a computer on the cluster (vide infra)
* Wait times: a good cluster is always busy. Expect a waiting time betwween 1h and 2 days until your calculation(s) start

=== Compute Workflow ===

A short description of the workflow how running calculation works can be found under [[Running Calculations]] and can give you a general idea.

=== Software ===

Basic usage is very simple: you run <code> module load module_name </code> and use the software, but important documentation and examples are also built into the modules.

The usage of the software on the cluster is described in [[Environment Modules]]

== Still With Us? ==

If you feel your calculations meet the requirements and you will save a lot of time despite some learning overhead, proceed to the [[Registration]] page.