Helix/bwVisu/JupyterLab

JupyterLab is an integrated development environment (IDE) that provides a flexible and scalable interface for the Jupyter Notebook system. It supports interactive data science and scientific computing across over 40 programming languages (including Python, Julia, and R).

Python version

You can see the default python version by running python --version

1. Activate the Miniconda module
2. Create a virtual environment with a custom python version via conda:

   conda create --name <myenv> python=<python version>
   

3. Install the ipykernel package:

   conda install ipykernel
   

4. Register your virtual environment as custom kernel to Jupyter:

   python3 -m ipykernel install --user --name=<myKernel>
   

5. Select your newly created kernel in Jupyter

Python packages

Add packages via venv virtual environments

1. Open a terminal and type:

   python3 -m venv <myEnv>
   

2. Activate your environment:

   source <myEnv>/bin/activate
   

3. Install your packages:

   pip install -U pip
   pip install <mypackage>
   

4. Install the ipykernel package:

   pip install ipykernel
   

5. Register your virtual environment as custom kernel to Jupyter:

   python3 -m ipykernel install --user --name=<myKernel>
   

Add packages via miniconda

1. Load the miniconda module by clicking first on the blue hexagon icon on the left-hand side of Jupyter's start page and then on the "load" Button right of the entry for miniconda in the software module menu. Open a terminal and complete the miniconda setup with:

   source $MINICONDA_HOME/etc/profile.d/conda.sh 

2. Open a terminal and create a conda environment:

   conda create --name <myenv>
   

3. Activate your environment:

   conda activate <myenv>
   

4. Install your packages:

   conda install <mypackage>
   

5. Install the ipykernel package:

   conda install ipykernel
   

6. Register your virtual environment as custom kernel to Jupyter:

   python3 -m ipykernel install --user --name=<myKernel>
   

Multi-Language Support

JupyterLab supports over 40 programming languages including Python, R, Julia, and Scala. This is achieved through the use of different kernels, which are implementations of the Jupyter notebook environment for each language. You can switch between kernels easily, allowing you to use the best tool for a specific task.

Example of switching kernels in a notebook:

1. On the cluster:

   $ module load math/R
   $ R
   > install.packages('IRkernel')

2. On bwVisu:
   1. Start Jupyter App
   2. In left menu: load math/R
   3. Open Console:

   $ R
   > IRkernel::installspec(displayname = 'R 4.2')

   4. Start kernel 'R 4.2' as console or notebook

   
   

   Interactive Widgets

   JupyterLab supports interactive widgets that can create UI controls for interactive data visualization and manipulation within the notebooks. Example of using an interactive widget:

   from ipywidgets import IntSlider
   slider = IntSlider()
   display(slider)

   These widgets can be sliders, dropdowns, buttons, etc., which can be connected to Python code running in the backend.

   FAQ

   1. Q: My conda commands are interrupted with message 'Killed'. What can I do?

      A: Request more memory when starting Jupyter. In the job settings open 'Cluster Settings' and look for the option 'Memory / node'.

   2. Q: How can I navigate to my SDS@hd folder in the file browser?

      A: Open a terminal and set a symbolic link to your SDS@hd folder in your home directory. For example:

      $ cd $HOME
      $ mkdir sds-hd
      $ cd sds-hd
      $ ln -s /mnt/sds-hd/sd16a001 sd16a001

   3. Q: Jupyterlab doesn't let me in but asks for a password. A: Try using more memory for the job. If this doesn't help, try using the inkognito mode of your browser as the browser cache might be the problem.

   4. Q: I prefer VSCode over JupyterLab. Can I start a JupyterLab job and then connect with it via VSCode? A: This is possible but, if you are a Helix user as well, it is not recommended as it wastes bwVisu resources. Please start the job directly on Helix instead. You can follow these steps:

      • normal Slurm job with a Jupyter instance (jupyter lab --no-browser --ip=0.0.0.0 --port=somenumber) and connect to that? That way, you could also automate most of the process instead of having to log in to a web page.