DACHS/Jupyter

Short description of Jupyter

Jupyter (an acronym for Julia, Python and R) is a web application, allowing interactive programming and visualization in a browser. Jupyter uses so-called Jupyter-Notebooks to load and store the program, input data and it's output (including visualization) in a JSON-based file, allowing exchange between different implementations (like Visual Studio Code plus a Jupyter Extension) and specifically allowing incrementally editing using a version-control system like git.

We provide JupyterHub at https://dachs-jupyter.hs-esslingen.de/ as described below.

Access requirements

Access to Jupyter is limited to IP addresses from the BelWü network. All partners of DACHS are connected to BelWü, so if you are on your campus network (e.g. in your office or on the Campus WiFi) you should be able to connect without restrictions. Otherwise, You will see a hint about having to connect using VPN to Your home institution (and make sure, that all packets are routed through your home institution's VPN, and not in SPLIT tunneling mode, see Login page.

Prerequisites

1. Register as described on the Registration
2. Then login to DACHS via SSH at least once: This ensures that your home directory is setup properly on DACHS
3. Make sure you're connected to the VPN of your university (see above)

Login Process

After having logged into

 https://dachs-jupyter.hs-esslingen.de

You will need to specify the resources, as described in the next section.

Logout Process

Your Jupyter Notebook ist started as any other Slurm job. If you just close your web browser, the Slurm job will continue to run in the background until it hits the time limit. Therefore, you should explicitly stop your server or log out from JupyterHub so that

• your Slurm job is stopped, and your allocated resources do not count towards your personal and organizational usage quota.
• resources are returned so that they can be used by others.

To do this, got to File > Hub Control Panel and click Stop My Server. Alternatively, you can also log out via File > Log Out. When you log out, your Jupyter Notebook is also stored.

Selection of Compute Resources

After you logged in via bwIDM, you can select the resources for your Jupyter Notebook.

Preselected are the following values, but you can adjust them as needed:

• 1 CPU core
• 16 GB of RAM
• No GPU
• Job runtime of 30 minutes
• Load the jupyter/ai module (a virtual environment containing libraries typically needed for machine learning)
• Use the default JupyterHub reservation. From Monday to Friday, from 08:00 to 20:00 o'clock there are four nodes reserved for interactive JupyterHub use. You can still try to start a Juptyer Notebook outside of this time window, however, this job requests is treated as any other job, meaning if the cluster has no available resources the job cannot be started.

When you adjusted your resources, press start and the Notebook is started for you.

If you want to use a custom reservation, for example a reservation you previously requested, then un-check the box after Use default reservation and enter your custom reservation name and the token that you received from us. You can also try to start the Jupyter Notebook without a custom reservation (and without the default reservation) This job is treated as any other by Slurm, meaning it is scheduled if there are available resources.

Software Stacks

We provide two Python environments:

• jupyter/minimal: This Python environment contains just the basic packages to run the Jupyter notebook.

You can use this environment as base for you own environment or use environment that you completely build yourself.

• jupyter/ai (default): This Python environment is preselected in the resource selection dialog.

Additional to the minimal environment, it contains plotly, scikit-learn, seaborn, tensorflow, torchvision.

You can also build and use your own environment in a Jupyter notebbok. First you create the Python environment with your required dependencies. Then you make this environment available as kernel for the Jupyter notebook that you can select. Take a look at the bwUnicluster documentation

Request a Reservation for Your Lecture

Requesting a unique reservation for JupyterHub has the advantage that the requested resources will be available to you for sure at the time you need them. Furthermore, it's easy:

1. Send an email with the following information:

• Start time and duration
• How many nodes do you request? Generally, we assign nodes from the gpu1 partition to JupyterHub jobs.
• Do you request a one-time or periodic reservation?
• The usernames that should have access to the reservation. This is at least your username. (For example es_username). If you want the students of your lecture to have access as well, export a list of their usernames and send these as well. We need to know the usernames to assign the correct entitlement to access the resource.

2. Wait for a reply. You'll get a reservation name and a reservation token that must be entered in the resource selection dialog of JupyterHub.