BwUniCluster3.0/Jupyter - Revision history

I Tachakor: /* Virtual environments */

2026-10-05T19:46:17Z

Virtual environments

← Older revision		Revision as of 21:46, 5 October 2026
Line 227:		Line 227:
	uv add --directory <myEnv> <packageName>		uv add --directory <myEnv> <packageName>
	</pre>		</pre>

	When using <code>uv</code>, the environment does not have to be activated with <code>source</code>. Commands can be executed directly through <code>uv</code>, which automatically uses the corresponding environment.

	=== Usage of virtual environment ===		=== Usage of virtual environment ===
Line 261:		Line 259:
	--display-name "Python (myEnv)"		--display-name "Python (myEnv)"
	</pre>		</pre>

	After installing the kernel (and possibly refreshing the browser window), a button named "Python (myEnv)" is available in JupyterLab. The kernel can also be selected from the drop-down menu.

	'''Attention'''		'''Attention'''

I Tachakor: /* Virtual environments */

2026-10-05T19:35:48Z

Virtual environments

← Older revision		Revision as of 21:35, 5 October 2026
Line 209:		Line 209:

	=== Creation of virtual environment ===		=== Creation of virtual environment ===

			A virtual environment can be created using the standard Python <code>venv</code> module:

	<pre>		<pre>
	python -m venv <myEnv>		python -m venv <myEnv>
	source <myEnv>/bin/activate		source <myEnv>/bin/activate
	pip install <packageName>		pip install <packageName>
	deactivate		deactivate
	</pre>		</pre>

	The additional packages are installed under <code><myEnv>/lib/python3.11/site-packages/</code>.		The additional packages are installed under <code><myEnv>/lib/python3.11/site-packages/</code>.

			Alternatively, a virtual environment can be created and managed using <code>uv</code>:

			<pre>
			uv venv <myEnv>
			uv add --directory <myEnv> <packageName>
			</pre>

		⚫	When using <code>uv</code>, the environment does not have to be activated with <code>source</code>. Commands can be executed directly through <code>uv</code>, which automatically uses the corresponding environment.

	=== Usage of virtual environment ===		=== Usage of virtual environment ===

	In order to use ~~the~~ virtual environment, it has to be activated via <code>source <myEnv>/bin/activate</code>. <code>PYTHONPATH</code> is set accordingly. Deactivation of the venv is done via <code>deactivate</code>.		In order to use a virtual environment created with <code>venv</code>, it has to be activated via <code>source <myEnv>/bin/activate</code>. <code>PYTHONPATH</code> is set accordingly. Deactivation of the venv is done via <code>deactivate</code>.

			For environments managed by <code>uv</code>, activation is not required. Python and other commands can be executed using <code>uv run</code>, which automatically uses the environment managed by <code>uv</code>.

	=== Usage of virtual environment in JupyterLab ===		=== Usage of virtual environment in JupyterLab ===
Line 230:		Line 243:
	source <myEnv>/bin/activate		source <myEnv>/bin/activate
	python -m ipykernel install \		python -m ipykernel install \
	--user \		--user \
	--name myEnv \		--name myEnv \
	--display-name "Python (myEnv)"		--display-name "Python (myEnv)"
	</pre>		</pre>

Line 244:		Line 257:
	<pre>		<pre>
	uv run python -m ipykernel install \		uv run python -m ipykernel install \
	--user \		--user \
	--name myEnv \		--name myEnv \
	--display-name "Python (myEnv)"		--display-name "Python (myEnv)"
	</pre>		</pre>

Line 257:		Line 270:
	uv add ipykernel		uv add ipykernel
	</pre>		</pre>

⚫	When using <code>uv</code>, the environment does not have to be activated with <code>source</code> ~~before~~ ~~installing~~ ~~the~~ ~~kernel.~~ ~~The~~ <code>uv ~~run~~</code> ~~command~~ automatically ~~runs~~ ~~Python~~ ~~from the~~ environment ~~managed by <code>uv</code>~~.

	== Examples on Data processing, Machine Learning & Visualization ==		== Examples on Data processing, Machine Learning & Visualization ==

I Tachakor: /* Usage of virtual environment in JupyterLab */

2026-10-05T18:54:42Z

Usage of virtual environment in JupyterLab

@@ Line 239: / Line 239: @@
 '''Attention'''
 The (Lmod) base module you used in the Creation of virtual environment step must be loaded to use the venv. However, to be on the safe side, you can also use the system Python (<code>/usr/bin/python3.11</code>) at creation time, which is available even without any <code>jupyter/{base,tensorflow}</code> module loaded.
 == Examples on Data processing, Machine Learning & Visualization ==

S Fischer: fixed broken links

2026-05-04T12:25:13Z

fixed broken links

← Older revision		Revision as of 14:25, 4 May 2026
Line 171:		Line 171:
	* <code>jupyter/base</code>		* <code>jupyter/base</code>
	*: Basic installation of JupyterLab.		*: Basic installation of JupyterLab.
	*: For a complete list of pre-installed packages, please refer to [https://~~uc2~~-jupyter.scc.kit.edu/software-modules/#pre-installed-software-packages this site].		*: For a complete list of pre-installed packages, please refer to [https://uc3-jupyter.scc.kit.edu/software-modules/#pre-installed-software-packages this site].

	* <code>jupyter/tensorflow</code> (default at login, will be deprecated with the advent of bwUniCluster 3.0)		* <code>jupyter/tensorflow</code> (default at login, will be deprecated with the advent of bwUniCluster 3.0)
	*: Preinstalled software packages for machine learning applications. Includes among others TensorFlow, Keras, Torch, Pandas, Matplotlib, SKLearn.		*: Preinstalled software packages for machine learning applications. Includes among others TensorFlow, Keras, Torch, Pandas, Matplotlib, SKLearn.
	*: For a complete list of pre-installed packages and their respective version, please refer to [https://~~uc2~~-jupyter.scc.kit.edu/software-modules/#pre-installed-software-packages this site].		*: For a complete list of pre-installed packages and their respective version, please refer to [https://uc3-jupyter.scc.kit.edu/software-modules/#pre-installed-software-packages this site].

	* <code>jupyter/ai</code> (will be the new default at login für bwUniCluster 3.0, contains all latest and greatest software for AI workflows)		* <code>jupyter/ai</code> (will be the new default at login für bwUniCluster 3.0, contains all latest and greatest software for AI workflows)
	*: Preinstalled software packages for machine learning applications. Includes among others TensorFlow, Keras, Torch, Torchvision, Lighning, Pandas, Matplotlib, SKLearn.		*: Preinstalled software packages for machine learning applications. Includes among others TensorFlow, Keras, Torch, Torchvision, Lighning, Pandas, Matplotlib, SKLearn.
	*: For a complete list of pre-installed packages and their respective version, please refer to [https://~~uc2~~-jupyter.scc.kit.edu/software-modules/#pre-installed-software-packages this site].		*: For a complete list of pre-installed packages and their respective version, please refer to [https://uc3-jupyter.scc.kit.edu/software-modules/#pre-installed-software-packages this site].

	* <code>jupyter/extensions</code>		* <code>jupyter/extensions</code>

M Carmesin: /* Julia language */

2025-05-07T07:44:51Z

Julia language

← Older revision		Revision as of 09:44, 7 May 2025
Line 261:		Line 261:
	== Julia language ==		== Julia language ==

	In order to use Julia language in JupyterLab, the Lmod module <code>~~devel~~/julia/1.6.2</code> has to be loaded (blue button in JupyterLab or <code>module ~~devel~~/julia/1.6.2</code> in terminal) ~~and~~ a corresponding kernel has to be ~~installed.~~		In order to use Julia language in JupyterLab, the Lmod module <code>math/julia/1.10.8</code> has to be loaded (blue button in JupyterLab or <code>module math/julia/1.10.8</code> in terminal). When the module is loaded from the JupytherLab UI, the corresponding kernel will be installed. If you use the terminal, this has to be done manually

	<pre>		<pre>
Line 269:		Line 269:
	</pre>		</pre>

	After installing the kernel , a button named "Julia 1.6.2" is available in JupyterLab. The kernel can also be selected from the drop-down menu.		After installing the kernel, a button named "Julia 1.10.8" is available in JupyterLab. The kernel can also be selected from the drop-down menu.

	'''Attention:'''		'''Attention:'''
	Don't forget to load the <code>~~devel~~/julia/1.6.2</code> module (blue button) before using the kernel.		Don't forget to load the <code>math/julia/1.10.8</code> module (blue button) before using the kernel.

	= Jupyter Container Mode =		= Jupyter Container Mode =

P Schuhmacher: /* Installation of further software */

2025-04-03T13:45:02Z

Installation of further software

← Older revision		Revision as of 15:45, 3 April 2025
Line 197:		Line 197:
	<li>User-Installation (not recommended)<br>		<li>User-Installation (not recommended)<br>
	<code>pip install --user <packageName> </code><br>		<code>pip install --user <packageName> </code><br>
	The additional packages are installed under $HOME/.local/lib/python3.9/site-packages/ which is part of PYTHONPATH.		The additional packages are installed under $HOME/.local/lib/python3.11/site-packages/ which is part of PYTHONPATH.
	</li>		</li>
	<li>Virtual environments (recommended)<br>		<li>Virtual environments (recommended)<br>
Line 217:		Line 217:
	</pre>		</pre>

	The additional packages are installed under <code><myEnv>/lib/python3.9/site-packages/</code>.		The additional packages are installed under <code><myEnv>/lib/python3.11/site-packages/</code>.

	=== Usage of virtual environment ===		=== Usage of virtual environment ===
Line 238:		Line 238:

	'''Attention'''		'''Attention'''
	The (Lmod) base module you used in the Creation of virtual environment step must be loaded to use the venv. However, to be on the safe side, you can also use the system Python (<code>/usr/bin/python3.9</code>) at creation time, which is available even without any <code>jupyter/{base,tensorflow}</code> module loaded.		The (Lmod) base module you used in the Creation of virtual environment step must be loaded to use the venv. However, to be on the safe side, you can also use the system Python (<code>/usr/bin/python3.11</code>) at creation time, which is available even without any <code>jupyter/{base,tensorflow}</code> module loaded.

	== Examples on Data processing, Machine Learning & Visualization ==		== Examples on Data processing, Machine Learning & Visualization ==

P Schuhmacher: /* Jupyter Container Mode */

2025-04-03T13:44:11Z

Jupyter Container Mode

← Older revision		Revision as of 15:44, 3 April 2025
Line 283:		Line 283:

	'''Attention:'''		'''Attention:'''
	Make sure Python3.9 and pip are installed in the Container or the notebook will not spawn. This can be checked via command python3.9 -m pip list inside your container		Make sure Python3.11 and pip are installed in the Container or the notebook will not spawn. This can be checked via command python3.11 -m pip list inside your container

	It is advised to create the container prior via e.G. enroot and install all necessary software. For more information see [https://wiki.bwhpc.de/e/BwUniCluster2.0/Containers#SLURM_Integration here]		It is advised to create the container prior via e.G. enroot and install all necessary software. For more information see [https://wiki.bwhpc.de/e/BwUniCluster2.0/Containers#SLURM_Integration here]

P Schuhmacher at 13:42, 3 April 2025

2025-04-03T13:42:48Z

← Older revision		Revision as of 15:42, 3 April 2025
Line 78:		Line 78:

	== Prioritized access to computing resources on bwUniCluster 3.0 ==		== Prioritized access to computing resources on bwUniCluster 3.0 ==
	The use of Jupyter requires the immediate availability of computing resources since the JupyterLab server is started within an interactive Slurm session. To improve the availability of CPUs/GPUs for interactive supercomputing with Jupyter, '''automatic reservation''' for CPU (~~single~~) and GPU (~~gpu_8~~) resources has been set up on '''bwUniCluster 3.0'''. It is active '''between 8am and 8pm''' every weekday. The reservation is automatically active if		The use of Jupyter requires the immediate availability of computing resources since the JupyterLab server is started within an interactive Slurm session. To improve the availability of CPUs/GPUs for interactive supercomputing with Jupyter, '''automatic reservation''' for CPU (cpu_il) and GPU (gpu_a100_il) resources has been set up on '''bwUniCluster 3.0'''. It is active '''between 8am and 8pm''' every weekday. The reservation is automatically active if

	* no other reservation is set manually		* no other reservation is set manually

S Braun at 07:48, 25 February 2025

2025-02-25T07:48:01Z

← Older revision		Revision as of 09:48, 25 February 2025
Line 274:		Line 274:
	Don't forget to load the <code>devel/julia/1.6.2</code> module (blue button) before using the kernel.		Don't forget to load the <code>devel/julia/1.6.2</code> module (blue button) before using the kernel.

			= Jupyter Container Mode =

			The container integration on the jupyterhub is done via pyxis. In order to use it the checkmark for container mode has to be clicked. Available options are:

			* <code> --container-image:</code> The container image to use. Corresponds to the pyxis option --container-image
			* <code> --container-name:</code> The name of the image to use. Corresponds to the pyxis option --container-name. Already downloaded containers in ~/.local/share/enroot can be startet by simply specifing their name.
			* <code> --container-mount-home:</code> Corresponds to the pyxis option --container-mount-home. Mounts the home-directory

			'''Attention:'''
			Make sure Python3.9 and pip are installed in the Container or the notebook will not spawn. This can be checked via command python3.9 -m pip list inside your container

			It is advised to create the container prior via e.G. enroot and install all necessary software. For more information see [https://wiki.bwhpc.de/e/BwUniCluster2.0/Containers#SLURM_Integration here]

	----		----

S Braun at 12:30, 9 January 2025

2025-01-09T12:30:40Z

← Older revision		Revision as of 14:30, 9 January 2025
Line 22:		Line 22:
	\|style="padding:5px; background:#cef2e0; text-align:left"\|		\|style="padding:5px; background:#cef2e0; text-align:left"\|
	Access to Jupyter is '''limited to IP addresses from the BelWü network'''.		Access to Jupyter is '''limited to IP addresses from the BelWü network'''.
	All home institutions of our current users are connected to BelWü, so if you are on your campus network (e.g. in your office or on the Campus WiFi) you should be able to connect to bwUniCluster 2.0 without restrictions.		All home institutions of our current users are connected to BelWü, so if you are on your campus network (e.g. in your office or on the Campus WiFi) you should be able to connect to bwUniCluster 3.0 without restrictions.
	If you are outside one of the BelWü networks (e.g. at home), a VPN connection to the home institution or a connection to an SSH jump host at the home institution must be established first.		If you are outside one of the BelWü networks (e.g. at home), a VPN connection to the home institution or a connection to an SSH jump host at the home institution must be established first.
	\|}		\|}

	To use Jupyter on the HPC resources of SCC, the access requirements for [https://wiki.bwhpc.de/e/Registration/bwUniCluster bwUniCluster 2.0] apply. A [https://wiki.bwhpc.de/e/Registration/bwUniCluster registration] is required. Please note, You should've completed registration and tested your login once using [https://wiki.bwhpc.de/e/Registration/SSH Secure Shell (ssh)].		To use Jupyter on the HPC resources of SCC, the access requirements for [https://wiki.bwhpc.de/e/Registration/bwUniCluster bwUniCluster 3.0] apply. A [https://wiki.bwhpc.de/e/Registration/bwUniCluster registration] is required. Please note, You should've completed registration and tested your login once using [https://wiki.bwhpc.de/e/Registration/SSH Secure Shell (ssh)].

	= Login process =		= Login process =

	Login takes place at		Login takes place at
	* bwUniCluster 2.0: [https://~~uc2~~-jupyter.scc.kit.edu ~~uc2~~-jupyter.scc.kit.edu]		* bwUniCluster 3.0: [https://uc3-jupyter.scc.kit.edu uc3-jupyter.scc.kit.edu]
	* SDIL: [https://sdil-jupyter.scc.kit.edu sdil-jupyter.scc.kit.edu]		* SDIL: [https://sdil-jupyter.scc.kit.edu sdil-jupyter.scc.kit.edu]
	* HoreKa: [https://hk-jupyter.scc.kit.edu hk-jupyter.scc.kit.edu]		* HoreKa: [https://hk-jupyter.scc.kit.edu hk-jupyter.scc.kit.edu]
Line 77:		Line 77:
	The spawning timeout is currently set to 10 minutes. With a normal workload of the HPC facility, this time is usually sufficient to get interactive resources.		The spawning timeout is currently set to 10 minutes. With a normal workload of the HPC facility, this time is usually sufficient to get interactive resources.

	== Prioritized access to computing resources on bwUniCluster 2.0 ==		== Prioritized access to computing resources on bwUniCluster 3.0 ==
	The use of Jupyter requires the immediate availability of computing resources since the JupyterLab server is started within an interactive Slurm session. To improve the availability of CPUs/GPUs for interactive supercomputing with Jupyter, '''automatic reservation''' for CPU (single) and GPU (gpu_8) resources has been set up on '''bwUniCluster 2.0'''. It is active '''between 8am and 8pm''' every weekday. The reservation is automatically active if		The use of Jupyter requires the immediate availability of computing resources since the JupyterLab server is started within an interactive Slurm session. To improve the availability of CPUs/GPUs for interactive supercomputing with Jupyter, '''automatic reservation''' for CPU (single) and GPU (gpu_8) resources has been set up on '''bwUniCluster 3.0'''. It is active '''between 8am and 8pm''' every weekday. The reservation is automatically active if

	* no other reservation is set manually		* no other reservation is set manually
Line 188:		Line 188:
	A continuously updated list with the installed packages can be found on the corresponding subpage of the respective cluster:		A continuously updated list with the installed packages can be found on the corresponding subpage of the respective cluster:

	* bwUniCluster 2.0: [https://~~uc2~~-jupyter.scc.kit.edu/software-modules ~~uc2~~-jupyter.scc.kit.edu/software-modules]		* bwUniCluster 3.0: [https://uc3-jupyter.scc.kit.edu/software-modules uc3-jupyter.scc.kit.edu/software-modules]
	* HoreKa: [https://hk-jupyter.scc.kit.edu/software-modules hk-jupyter.scc.kit.edu/software-modules]		* HoreKa: [https://hk-jupyter.scc.kit.edu/software-modules hk-jupyter.scc.kit.edu/software-modules]