Energy Efficient Cluster Usage: Difference between revisions
P Weisbrod (talk | contribs) |
(Small spelling changes and update the estimates of power consumption) |
||
| Line 3: | Line 3: | ||
You are all aware of the rising energy costs and you are certainly careful to economize your energy |
You are all aware of the rising energy costs and you are certainly careful to economize your energy |
||
consumption at home. |
consumption at home. |
||
But are you aware |
But are you aware of the energy consumed by your computational jobs? |
||
An average compute job running on just a single node for one day |
An average compute job running on just a single node for one day may easily consume 10 kWh or |
||
even more. |
even more. |
||
| Line 15: | Line 15: | ||
Besides energy costs, even this single job alone |
Besides energy costs, even this single job alone |
||
will also contribute to climate change by adding |
will also contribute to climate change by adding |
||
around 5 kg of CO |
around 5 kg of CO<sub>2</sub> to the atmosphere (based on the average German power mix) |
||
equivalent to driving a distance of 30 km by car. |
which is roughly equivalent to driving a distance of 30 km by car. |
||
You get the point: Please always keep this in mind |
You get the point: Please always keep this in mind |
||
| Line 36: | Line 36: | ||
* Run only necessary jobs: Please consider testing new setups and their output for validity prior to submitting a huge amount of similar jobs |
* Run only necessary jobs: Please consider testing new setups and their output for validity prior to submitting a huge amount of similar jobs |
||
* Start small: Run Your problem on a small amount of parallel entities (be it processes or threads) first |
* Start small: Run Your problem on a small amount of parallel entities (be it processes or threads) first |
||
* Estimate the runtime of the parallel job as exactly as possible to increase efficiency of the scheduling of the whole system |
|||
* Use the proper tools for development: If You develop your own code, please use the proper tools for debugging and parallel performance analysis. More information is available on the bwHPC Wiki. |
* Use the proper tools for development: If You develop your own code, please use the proper tools for debugging and parallel performance analysis. More information is available on the bwHPC Wiki. |
||
* A look at the job feedback can help you determine if you are using the cluster efficiently |
* A look at the job feedback can help you determine if you are using the cluster efficiently |
||
Revision as of 11:03, 22 November 2022
General Issue with Energy Efficiency
You are all aware of the rising energy costs and you are certainly careful to economize your energy consumption at home. But are you aware of the energy consumed by your computational jobs? An average compute job running on just a single node for one day may easily consume 10 kWh or even more.
That translates roughly to one of the following activities:
- toasting about 1330 slices of toast in a toaster
- continuously blow-drying your hair for about 10 hours
- actively working on a laptop for about 500 hours
- brewing 700 cups of coffee
Besides energy costs, even this single job alone will also contribute to climate change by adding around 5 kg of CO2 to the atmosphere (based on the average German power mix) which is roughly equivalent to driving a distance of 30 km by car.
You get the point: Please always keep this in mind when submitting tens or even hundreds of jobs to the queue, just like you do when switching on your electrical devices at home. Also, please always think carefully about how many resources your jobs really need and whether your application really benefits from allocating more cores for the jobs. Application speedup is often limited and does not scale linearly with the number of dedicated cores. But energy consumption usually does ...
Using as many resources as possible does not make a power user. Using them wisely does. If in doubt, just ask.
General recommendations
- Choose the most efficient algorithms for the given problem
- Run only necessary jobs: Please consider testing new setups and their output for validity prior to submitting a huge amount of similar jobs
- Start small: Run Your problem on a small amount of parallel entities (be it processes or threads) first
- Estimate the runtime of the parallel job as exactly as possible to increase efficiency of the scheduling of the whole system
- Use the proper tools for development: If You develop your own code, please use the proper tools for debugging and parallel performance analysis. More information is available on the bwHPC Wiki.
- A look at the job feedback can help you determine if you are using the cluster efficiently