Helix/bwVisu/JupyterLab

JupyterLab is an integrated development environment (IDE) that provides a flexible and scalable interface for the Jupyter Notebook system. It supports interactive data science and scientific computing across over 40 programming languages (including Python, Julia, and R).

Python version

You can see the default python version by running python --version

1. Activate the Miniconda module
2. Create a virtual environment with a custom python version via conda:

   conda create --name <myenv> python=<python version>
   

3. Install the ipykernel package:

   conda install ipykernel
   

4. Register your virtual environment as custom kernel to Jupyter:

   python3 -m ipykernel install --user --name=<myKernel>
   

5. Select your newly created kernel in Jupyter

Python packages

Add packages via venv virtual environments

1. Open a terminal and type:

   python3 -m venv <myEnv>
   

2. Activate your environment:

   source <myEnv>/bin/activate
   

3. Install your packages:

   pip install -U pip
   pip install <mypackage>
   

4. Install the ipykernel package:

   pip install ipykernel
   

5. Register your virtual environment as custom kernel to Jupyter:

   python3 -m ipykernel install --user --name=<myKernel>
   

Add packages via conda environments

1. Load the miniforge module by clicking first on the blue hexagon icon on the left-hand side of Jupyter's start page and then on the "load" button right of the entry for miniforge in the software module menu.
2. Open a terminal and create a conda environment:

   conda create --name <myenv>
   

3. Activate your environment:

   conda activate <myenv>
   

4. Install your packages:

   conda install <mypackage>
   

5. Install the ipykernel package:

   conda install ipykernel
   

6. Register your virtual environment as custom kernel to Jupyter:

   python3 -m ipykernel install --user --name=<myKernel>
   

Interactive Widgets

JupyterLab supports interactive widgets that can create UI controls for interactive data visualization and manipulation within the notebooks. Example of using an interactive widget:

from ipywidgets import IntSlider
slider = IntSlider()
display(slider)

These widgets can be sliders, dropdowns, buttons, etc., which can be connected to Python code running in the backend.

Python kernels

Python kernels are implementations of the Jupyter notebook environment for different languages or virtual environments. You can switch between kernels easily, allowing you to use the best tool for a specific task.

By default, custom kernels are installed to ~/.local/share/jupyter/kernels/.

Shared use of conda environments

Follow these instructions if you want to share conda environments with your colleagues:

1. Create the environment:
   • Create a shared directory (optimally on SDS@hd) into wich the ipython kernels are installed, for example "conda_kernels". Never share your home directory.
   • Create a conda environment and register it as kernel.
   • Optional: You can add another shared directory, for example "conda_envs", to share not just the kernels but also the underlying conda environments. A conda environment can be saved as a yml file.
2. Use the shared environment:
   • After having the conda environment installed as a jupyter kernel, JupyterLab needs be able to detect the kernel. Therefore, the path to the kernel must be added to the list of juypter data directories. This can be done by setting a specific environment variable in your .bashrc file. The file can be edited via a terminal or with the bwVisu file viewer.

     export JUPYTER_DATA_DIR=<path_to_shared_directory>
     

   • Optional: You can check the jupyter paths by running the following line in a terminal within JupyterLab or on Helix. The latter needs Jupyter to be installed.

     jupyter --paths
     

   • Start a bwVisu JupyterLab Job. You can find the new environment under "Kernels".

Multi-Language Support

JupyterLab supports over 40 programming languages including Python, R, Julia, and Scala. This is achieved through the use of different kernels.

R Kernel

1. On the cluster:

   $ module load math/R
   $ R
   > install.packages('IRkernel')

2. On bwVisu:
   1. Start Jupyter App
   2. In left menu: load math/R
   3. Open Console:

   $ R
   > IRkernel::installspec(displayname = 'R 4.2')

   4. Start kernel 'R 4.2' as console or notebook

Julia Kernel

Load the math/julia module. Open the Terminal.

julia
]
add IJulia

After that, Julia is available as a kernel.

FAQ

1. My conda commands are interrupted with message 'Killed'.
   Request more memory when starting Jupyter.
2. How can I navigate to my SDS@hd folder in the file browser?
   Open a terminal and set a symbolic link to your SDS@hd folder in your home directory. For example:

cd $HOME
mkdir sds-hd
cd sds-hd
ln -s /mnt/sds-hd/sd16a001 sd16a001

3. Jupyterlab doesn't let me in but asks for a password.
   Try using more memory for the job. If this doesn't help, try using the inkognito mode of your browser as the browser cache might be the problem.
4. I prefer VSCode over JupyterLab. Can I start a JupyterLab job and then connect with it via VSCode?
   This is possible but, if you are a bwForCluster Helix user as well, it is not recommended as it wastes bwVisu resources. Please start the job directly on Helix instead. You can find the instructions at the VSCode page.