Helix/bwVisu/JupyterLab

JupyterLab is an integrated development environment (IDE) that provides a flexible and scalable interface for the Jupyter Notebook system. It supports interactive data science and scientific computing across over 40 programming languages (including Python, Julia, and R).

Change Python Version

The default python version can be seen by running python --version in the terminal.

A different python version can be installed into a new virtual environment and then registered as IPython kernel for the usage in JupyterLab. This is explained in the chapter add packages via conda environments.

Install Python Packages

Python packages can be added by installing them into a virtual environment and then creating an IPython kernel from the virtual environment.
Kernels can be shared. See the notes below.
If you want to move a virtual environment, it is adivsed to recreate it in the new place. Otherwise, dependencies based on relative paths will break.

1. Create a virtual environment with...
   • ...venv or
   • ...conda (choose this option if you want to install a different python version) or
   • ...uv if you want to install a different python version but don't want to use conda.
2. Create an IPython kernel from the virtual environment
3. Use the kernel within JupyterLab
   • By default new kernels are saved under ~/.local/share/jupyter and this location is automatically detected. Therefore, new kernels are directly available.
   • If the kernel is saved somewhere else, the path can be provided in the "Kernel path" field when submitting the job. For a kernel placed under path_to_parent_dir/share/jupyter/kernels/my_kernel the needed path would be path_to_parent_dir.

Notes regarding the sharing of IPython kernels

• The virtual environment and the kernel need to be placed in a shared directory. For example at SDS@hd.
• There could be a subdirectory for the virtual environments and one for the kernels.
• The path to the kernels is saved in the environment variable $JUPYTER_DATA_DIR. Jupyter relevant paths can be seen with jupyter --paths

Add packages via venv virtual environments

More information about venv or other python virtual environments can be found at the Python page.

Steps for creating a venv virtual environment:

1. Open a terminal.
2. Create a new virtual evironment:

   python3 -m venv <env_parent_dir>/<env_name>
   

   • env_parent_dir is the path to the folder where the virtual environment shall be created. Relative paths can be used.
   • Caution: If you you want to share the environment with others, make sure to already create it in the shared place.
3. Activate the environment:

   source <env_parent_dir>/<env_name>/bin/activate
   

4. Update pip and install packages:

   pip install -U pip
   pip install <packagename>
   

5. Create an IPython kernel

Add packages via Conda virtual environments

More information about using conda can be found at the Conda page.

1. Load the miniforge module by clicking first on the blue hexagon icon on the left-hand side of Jupyter's start page and then on the "load" button right of the entry for miniforge in the software module menu.
2. Open a terminal.
3. Create a new virtual environment:
   • If you are the only person using the environment, you can install it in your home directory:

   conda create --name <env_name> python=<python version>
   

   • If you want to install it into a different directory, for example a shared place:

   conda create --prefix <path_to_shared_directory>/<env_name> python=<python version>
   

4. Activate your environment:

   conda activate <myenv>
   

5. Install your packages:

   conda install <mypackage>
   

6. Create an IPython kernel

Create an IPython Kernel

Python kernels are implementations of the Jupyter notebook environment for different languages or virtual environments. You can switch between kernels easily, allowing you to use the best tool for a specific task. conda_kernels

Create a kernel from a virtual environment

1. Activate the virtual environment.
2. Install the ipykernel package:

   pip install ipykernel
   

3. Register the virtual environment as custom kernel to Jupyter.
   • If you are the only person using the environment:

   python3 -m ipykernel install --user --name=<kernel_name>
   

   The kernel can be found under ~/.local/share/jupyter/kernels/.

   • If you installed the environment in a shared place and want to have the kernel there as well:

   python3 -m ipykernel install --prefix <path_to_kernel_folder> --name=<kernel_name>
   

   The kernel can then be found under path_to_kernel_folder/share/jupyter/kernels/<kernel_name>. As long as the same path_to_kernel_folder is used, all kernels will be saved next to each other in "kernels".

Multi-Language Support

JupyterLab supports over 40 programming languages including Python, R, Julia, and Scala. This is achieved through the use of different kernels.

R Kernel

1. On the cluster:

   $ module load math/R
   $ R
   > install.packages('IRkernel')

2. On bwVisu:
   1. Start Jupyter App
   2. In left menu: load math/R
   3. Open Console:

   $ R
   > IRkernel::installspec(displayname = 'R 4.2')

   4. Start kernel 'R 4.2' as console or notebook

Julia Kernel

Load the math/julia module. Open the Terminal.

julia
]
add IJulia

After that, Julia is available as a kernel.

Interactive Widgets

JupyterLab supports interactive widgets that can create UI controls for interactive data visualization and manipulation within the notebooks. Example of using an interactive widget:

from ipywidgets import IntSlider
slider = IntSlider()
display(slider)

These widgets can be sliders, dropdowns, buttons, etc., which can be connected to Python code running in the backend.

FAQ

1. My conda commands are interrupted with message 'Killed'.
   Request more memory when starting Jupyter.
2. How can I navigate to my SDS@hd folder in the file browser?
   Open a terminal and set a symbolic link to your SDS@hd folder in your home directory. For example:

cd $HOME
mkdir sds-hd
cd sds-hd
ln -s /mnt/sds-hd/sd16a001 sd16a001

3. Jupyterlab doesn't let me in but asks for a password.
   Try using more memory for the job. If this doesn't help, try using the inkognito mode of your browser as the browser cache might be the problem.
4. I prefer VSCode over JupyterLab. Can I start a JupyterLab job and then connect with it via VSCode?
   This is not possible. Please start the job directly on Helix instead. You can find the instructions at the VSCode page.