Difference between revisions of "BwUniCluster2.0/Software/Python Dask"
< BwUniCluster2.0 | Software
(→Using Dask) |
|||
Line 8: | Line 8: | ||
== Using Dask == |
== Using Dask == |
||
− | In a new interactive shell, |
+ | In a new interactive shell, execute the following commands in Python: |
<pre> |
<pre> |
||
>>> from dask_jobqueue import SLURMCluster |
>>> from dask_jobqueue import SLURMCluster |
||
− | >>> cluster = SLURMCluster(cores= |
+ | >>> cluster = SLURMCluster(cores=X, memory='X GB', queue='X') |
− | >>> |
||
</pre> |
</pre> |
||
+ | You have to specify how many cores and memory you want for one dask worker. |
||
+ | |||
+ | <pre> |
||
+ | >>> cluster.scale (X) |
||
+ | </pre> |
||
+ | |||
+ | Replace |
||
---- |
---- |
||
[[Category:bwUniCluster_2.0|Access]][[Category:Access|bwUniCluster 2.0]] |
[[Category:bwUniCluster_2.0|Access]][[Category:Access|bwUniCluster 2.0]] |
Revision as of 10:49, 19 April 2020
This guide explains how to use Python Dask and dask-jobqueue on bwUniCluster2.0.
1 Installation
Use on of our pre-configured Python modules and load them with 'module load ...'. You have to install the packages 'dask' and 'das-jobqueue' if your are yousing an own conda environment.
2 Using Dask
In a new interactive shell, execute the following commands in Python:
>>> from dask_jobqueue import SLURMCluster >>> cluster = SLURMCluster(cores=X, memory='X GB', queue='X')
You have to specify how many cores and memory you want for one dask worker.
>>> cluster.scale (X)
Replace