bwHPC Wiki - User contributions [en]

BwUniCluster2.0/Maintenance/2021-10

2021-10-06T08:03:41Z

H Obermaier: /* Compilers, Libaries and Runtime Environments */

The following changes are currently planned for the maintenance interval starting on 11.10.2021 (Monday) at 8 AM and ending on 15.10.2020 (Friday) at 12 AM.

This list will be extended and specified in the next days and weeks and is subject to change

= Hardware =

* The Broadwell Login nodes (uc1e.scc.kit.edu) will be taken out of service permanently due to ongoing hardware and software issues and low utilisation. It is still possible to generate code for the Broadwell nodes using the compilers installed on the normal login nodes, or by starting an interactive job on the Broadwell compute nodes.

* All firmware versions on all components will be upgraded.

= Operating system =

* The operating system version will continue to be based on Red Hat Enterprise Linux (RHEL) 8.2.

= Compilers, Libaries and Runtime Environments =

* The modules below toolkit/oneAPI will be integrated into the normal module structure.
* The OneAPI versions 2021.1.0 to 2021.3.0 are replaced by the current version 2021.4.0.
* The obsolete Intel compiler 18.0 is removed.
* This will leave the Intel Compiler versions 19.0, 19.1 and 2021.4.0.
* The latest version, namely 2021.4.0, becomes the default compiler.
* There are two editions of the Intel Compiler version 2021.4.0.
** The so-called Classic Compilers are based on the previous Intel compilers. They are available via the module <code>compiler/intel/2021.4.0</code>.
** The next generation of Intel compilers is based on LLVM and will replace the Classic compilers according to Intel plans (see [https://software.intel.com/content/www/us/en/develop/blogs/adoption-of-llvm-complete-icx.html Intel C/C++ compilers complete adoption of LLVM]). They are available via the module <code>compiler/intel/2021.4.0_llvm</code>.

= Development tools =

= Userspace tools =

* The workspace tools will be switched to a different, more modern implementation. The workspace management commands will not change, and all existing workspaces will be converted automatically.

= Software Modules =

The following list it not exhaustive.

* Update math/R software module to latest version 4.1.x.

= Batch system =

= Storage =

* Upgrade of the Lustre file systems

= Graphics stack =

* The NVIDIA driver will be updated to the most recent version available.

= Containers =

= JupyterHub =

* The JupyterHub version will be updated to a more recent version.

BwUniCluster2.0/Maintenance/2021-10

2021-10-06T08:03:05Z

H Obermaier: /* Compilers, Libaries and Runtime Environments */

The following changes are currently planned for the maintenance interval starting on 11.10.2021 (Monday) at 8 AM and ending on 15.10.2020 (Friday) at 12 AM.

This list will be extended and specified in the next days and weeks and is subject to change

= Hardware =

* The Broadwell Login nodes (uc1e.scc.kit.edu) will be taken out of service permanently due to ongoing hardware and software issues and low utilisation. It is still possible to generate code for the Broadwell nodes using the compilers installed on the normal login nodes, or by starting an interactive job on the Broadwell compute nodes.

* All firmware versions on all components will be upgraded.

= Operating system =

* The operating system version will continue to be based on Red Hat Enterprise Linux (RHEL) 8.2.

= Compilers, Libaries and Runtime Environments =

* The modules below toolkit/oneAPI will be integrated into the normal module structure.
* The OneAPI versions 2021.1.0 to 2021.3.0 are replaced by the current version 2021.4.0.
* The obsolete Intel compiler 18.0 is removed.
* This will leave the Intel Compiler versions 19.0, 19.1 and 2021.4.0.
* The latest version, namely 2021.4.0, becomes the default compiler.
* There are two editions of the Intel Compiler version 2021.4.0.
* The so-called Classic Compilers are based on the previous Intel compilers. They are available via the module <code>compiler/intel/2021.4.0</code>.
* The next generation of Intel compilers is based on LLVM and will replace the Classic compilers according to Intel plans (see [https://software.intel.com/content/www/us/en/develop/blogs/adoption-of-llvm-complete-icx.html Intel C/C++ compilers complete adoption of LLVM]). They are available via the module <code>compiler/intel/2021.4.0_llvm</code>.

= Development tools =

= Userspace tools =

* The workspace tools will be switched to a different, more modern implementation. The workspace management commands will not change, and all existing workspaces will be converted automatically.

= Software Modules =

The following list it not exhaustive.

* Update math/R software module to latest version 4.1.x.

= Batch system =

= Storage =

* Upgrade of the Lustre file systems

= Graphics stack =

* The NVIDIA driver will be updated to the most recent version available.

= Containers =

= JupyterHub =

* The JupyterHub version will be updated to a more recent version.

BwUniCluster2.0/Maintenance/2021-10

2021-10-06T08:02:24Z

H Obermaier: /* Compilers, Libaries and Runtime Environments */

The following changes are currently planned for the maintenance interval starting on 11.10.2021 (Monday) at 8 AM and ending on 15.10.2020 (Friday) at 12 AM.

This list will be extended and specified in the next days and weeks and is subject to change

= Hardware =

* The Broadwell Login nodes (uc1e.scc.kit.edu) will be taken out of service permanently due to ongoing hardware and software issues and low utilisation. It is still possible to generate code for the Broadwell nodes using the compilers installed on the normal login nodes, or by starting an interactive job on the Broadwell compute nodes.

* All firmware versions on all components will be upgraded.

= Operating system =

* The operating system version will continue to be based on Red Hat Enterprise Linux (RHEL) 8.2.

= Compilers, Libaries and Runtime Environments =

* The modules below toolkit/oneAPI will be integrated into the normal module structure.
* The OneAPI versions 2021.1.0 to 2021.3.0 are replaced by the current version 2021.4.0.
* The obsolete Intel compiler 18.0 is removed.
* This will leave the Intel Compiler versions 19.0, 19.1 and 2021.4.0.
* The latest version, namely 2021.4.0, becomes the default compiler.
* There are two editions of the Intel Compiler version 2021.4.0. The so-called Classic Compilers are based on the previous Intel compilers. They are available via the module <code>compiler/intel/2021.4.0</code>. The next generation of Intel compilers is based on LLVM and will replace the Classic compilers according to Intel plans (see [https://software.intel.com/content/www/us/en/develop/blogs/adoption-of-llvm-complete-icx.html Intel C/C++ compilers complete adoption of LLVM]). They are available via the module <code>compiler/intel/2021.4.0_llvm</code>.

= Development tools =

= Userspace tools =

* The workspace tools will be switched to a different, more modern implementation. The workspace management commands will not change, and all existing workspaces will be converted automatically.

= Software Modules =

The following list it not exhaustive.

* Update math/R software module to latest version 4.1.x.

= Batch system =

= Storage =

* Upgrade of the Lustre file systems

= Graphics stack =

* The NVIDIA driver will be updated to the most recent version available.

= Containers =

= JupyterHub =

* The JupyterHub version will be updated to a more recent version.

BwUniCluster2.0/Maintenance/2021-10

2021-10-06T08:01:58Z

H Obermaier: /* Compilers, Libaries and Runtime Environments */

The following changes are currently planned for the maintenance interval starting on 11.10.2021 (Monday) at 8 AM and ending on 15.10.2020 (Friday) at 12 AM.

This list will be extended and specified in the next days and weeks and is subject to change

= Hardware =

* The Broadwell Login nodes (uc1e.scc.kit.edu) will be taken out of service permanently due to ongoing hardware and software issues and low utilisation. It is still possible to generate code for the Broadwell nodes using the compilers installed on the normal login nodes, or by starting an interactive job on the Broadwell compute nodes.

* All firmware versions on all components will be upgraded.

= Operating system =

* The operating system version will continue to be based on Red Hat Enterprise Linux (RHEL) 8.2.

= Compilers, Libaries and Runtime Environments =

* The modules below toolkit/oneAPI will be integrated into the normal module structure.
* The OneAPI versions 2021.1.0 to 2021.3.0 are replaced by the current version 2021.4.0.
* The obsolete Intel compiler 18.0 is removed.
* This will leave the Intel Compiler versions 19.0, 19.1 and 2021.4.0.
* The latest version, namely 2021.4.0, becomes the default compiler.
* There are two editions of the Intel Compiler version 2021.4.0. The so-called Classic Compilers are based on the previous Intel compilers. They are available via the module <code>compiler/intel/2021.4.0</code>. The next generation of Intel compilers is based on LLVM and will replace the Classic compilers according to Intel plans (see [https://software.intel.com/content/www/us/en/develop/blogs/adoption-of-llvm-complete-icx.html Intel C/C++ compilers complete adoption of LLVM]). They are available via the module @compiler/intel/2021.4.0_llvm@.

= Development tools =

= Userspace tools =

* The workspace tools will be switched to a different, more modern implementation. The workspace management commands will not change, and all existing workspaces will be converted automatically.

= Software Modules =

The following list it not exhaustive.

* Update math/R software module to latest version 4.1.x.

= Batch system =

= Storage =

* Upgrade of the Lustre file systems

= Graphics stack =

* The NVIDIA driver will be updated to the most recent version available.

= Containers =

= JupyterHub =

* The JupyterHub version will be updated to a more recent version.

BwUniCluster2.0/Maintenance/2021-10

2021-10-06T07:59:09Z

H Obermaier: /* Compilers, Libaries and Runtime Environments */

The following changes are currently planned for the maintenance interval starting on 11.10.2021 (Monday) at 8 AM and ending on 15.10.2020 (Friday) at 12 AM.

This list will be extended and specified in the next days and weeks and is subject to change

= Hardware =

* The Broadwell Login nodes (uc1e.scc.kit.edu) will be taken out of service permanently due to ongoing hardware and software issues and low utilisation. It is still possible to generate code for the Broadwell nodes using the compilers installed on the normal login nodes, or by starting an interactive job on the Broadwell compute nodes.

* All firmware versions on all components will be upgraded.

= Operating system =

* The operating system version will continue to be based on Red Hat Enterprise Linux (RHEL) 8.2.

= Compilers, Libaries and Runtime Environments =

* The modules below toolkit/oneAPI will be integrated into the normal module structure.
* The OneAPI versions 2021.1.0 to 2021.3.0 are replaced by the current version 2021.4.0.
* The obsolete Intel compiler 18.0 is removed.
* This will leave the Intel Compiler versions 19.0, 19.1 and 2021.4.0.
* The latest version, namely 2021.4.0, becomes the default compiler.
* There are two editions of the Intel Compiler version 2021.4.0. The so-called Classic Compilers are based on the previous Intel compilers. They are available via the module @compiler/intel/2021.4.0@. The next generation of Intel compilers is based on LLVM and will replace the Classic compilers according to Intel plans (see [https://software.intel.com/content/www/us/en/develop/blogs/adoption-of-llvm-complete-icx.html Intel C/C++ compilers complete adoption of LLVM]). They are available via the module @compiler/intel/2021.4.0_llvm@.

= Development tools =

= Userspace tools =

* The workspace tools will be switched to a different, more modern implementation. The workspace management commands will not change, and all existing workspaces will be converted automatically.

= Software Modules =

The following list it not exhaustive.

* Update math/R software module to latest version 4.1.x.

= Batch system =

= Storage =

* Upgrade of the Lustre file systems

= Graphics stack =

* The NVIDIA driver will be updated to the most recent version available.

= Containers =

= JupyterHub =

* The JupyterHub version will be updated to a more recent version.

BwUniCluster2.0/Maintenance/2021-10

2021-10-06T07:57:03Z

H Obermaier: /* Compilers, Libaries and Runtime Environments */

The following changes are currently planned for the maintenance interval starting on 11.10.2021 (Monday) at 8 AM and ending on 15.10.2020 (Friday) at 12 AM.

This list will be extended and specified in the next days and weeks and is subject to change

= Hardware =

* The Broadwell Login nodes (uc1e.scc.kit.edu) will be taken out of service permanently due to ongoing hardware and software issues and low utilisation. It is still possible to generate code for the Broadwell nodes using the compilers installed on the normal login nodes, or by starting an interactive job on the Broadwell compute nodes.

* All firmware versions on all components will be upgraded.

= Operating system =

* The operating system version will continue to be based on Red Hat Enterprise Linux (RHEL) 8.2.

= Compilers, Libaries and Runtime Environments =

* The modules below toolkit/oneAPI will be integrated into the normal module structure.
* The OneAPI versions 2021.1.0 to 2021.3.0 are replaced by the current version 2021.4.0.
* The obsolete Intel compiler 18.0 is removed.
* This will leave the Intel Compiler versions 19.0, 19.1 and 2021.4.0.
* The latest version, namely 2021.4.0, becomes the default compiler.
* There are two editions of the Intel Compiler version 2021.4.0. The so-called Classic Compilers are based on the previous Intel compilers. They are available via the module compiler/intel/2021.4.0. The next generation of Intel compilers is based on LLVM and will replace the Classic compilers according to Intel plans (see [https://software.intel.com/content/www/us/en/develop/blogs/adoption-of-llvm-complete-icx.html Intel C/C++ compilers complete adoption of LLVM
]). They are available via the module compiler/intel/2021.4.0_llvm.

= Development tools =

= Userspace tools =

* The workspace tools will be switched to a different, more modern implementation. The workspace management commands will not change, and all existing workspaces will be converted automatically.

= Software Modules =

The following list it not exhaustive.

* Update math/R software module to latest version 4.1.x.

= Batch system =

= Storage =

* Upgrade of the Lustre file systems

= Graphics stack =

* The NVIDIA driver will be updated to the most recent version available.

= Containers =

= JupyterHub =

* The JupyterHub version will be updated to a more recent version.

BwUniCluster2.0/Maintenance/2021-10

2021-10-06T07:55:05Z

H Obermaier: /* Compilers, Libaries and Runtime Environments */

The following changes are currently planned for the maintenance interval starting on 11.10.2021 (Monday) at 8 AM and ending on 15.10.2020 (Friday) at 12 AM.

This list will be extended and specified in the next days and weeks and is subject to change

= Hardware =

* The Broadwell Login nodes (uc1e.scc.kit.edu) will be taken out of service permanently due to ongoing hardware and software issues and low utilisation. It is still possible to generate code for the Broadwell nodes using the compilers installed on the normal login nodes, or by starting an interactive job on the Broadwell compute nodes.

* All firmware versions on all components will be upgraded.

= Operating system =

* The operating system version will continue to be based on Red Hat Enterprise Linux (RHEL) 8.2.

= Compilers, Libaries and Runtime Environments =

* The modules below toolkit/oneAPI will be integrated into the normal module structure.
* The OneAPI versions 2021.1.0 to 2021.3.0 are replaced by the current version 2021.4.0.
* The obsolete Intel compiler 18.0 is removed.
* This will leave the Intel Compiler versions 19.0, 19.1 and 2021.4.0.
* The latest version, namely 2021.4.0, becomes the default compiler.
* There are two editions of the Intel Compiler version 2021.4.0.
The so-called Classic Compilers are based on the previous Intel compilers. They are available via the module compiler/intel/2021.4.0.
The next generation of Intel compilers is based on LLVM and will replace the Classic compilers according to Intel plans (see
https://software.intel.com/content/www/us/en/develop/blogs/adoption-of-llvm-complete-icx.html). They are available via the module compiler/intel/2021.4.0_llvm.

= Development tools =

= Userspace tools =

* The workspace tools will be switched to a different, more modern implementation. The workspace management commands will not change, and all existing workspaces will be converted automatically.

= Software Modules =

The following list it not exhaustive.

* Update math/R software module to latest version 4.1.x.

= Batch system =

= Storage =

* Upgrade of the Lustre file systems

= Graphics stack =

* The NVIDIA driver will be updated to the most recent version available.

= Containers =

= JupyterHub =

* The JupyterHub version will be updated to a more recent version.

BwUniCluster2.0/Maintenance/2021-10

2021-10-06T07:52:37Z

H Obermaier: /* Compilers, Libaries and Runtime Environments */

The following changes are currently planned for the maintenance interval starting on 11.10.2021 (Monday) at 8 AM and ending on 15.10.2020 (Friday) at 12 AM.

This list will be extended and specified in the next days and weeks and is subject to change

= Hardware =

* The Broadwell Login nodes (uc1e.scc.kit.edu) will be taken out of service permanently due to ongoing hardware and software issues and low utilisation. It is still possible to generate code for the Broadwell nodes using the compilers installed on the normal login nodes, or by starting an interactive job on the Broadwell compute nodes.

* All firmware versions on all components will be upgraded.

= Operating system =

* The operating system version will continue to be based on Red Hat Enterprise Linux (RHEL) 8.2.

= Compilers, Libaries and Runtime Environments =

* The modules below toolkit/oneAPI will be integrated into the normal module structure.
* The OneAPI versions 2021.1.0 to 2021.3.0 are replaced by the current version 2021.4.0.
* The obsolete Intel compiler 18.0 is removed.
* This will leave the Intel Compiler versions 19.0, 19.1 and 2021.4.0.

= Development tools =

= Userspace tools =

* The workspace tools will be switched to a different, more modern implementation. The workspace management commands will not change, and all existing workspaces will be converted automatically.

= Software Modules =

The following list it not exhaustive.

* Update math/R software module to latest version 4.1.x.

= Batch system =

= Storage =

* Upgrade of the Lustre file systems

= Graphics stack =

* The NVIDIA driver will be updated to the most recent version available.

= Containers =

= JupyterHub =

* The JupyterHub version will be updated to a more recent version.

BwUniCluster3.0/Software/Start vnc desktop

2017-08-17T12:01:20Z

H Obermaier: Created page with "The Linux 3D graphics stack is based on ''X11'' and ''OpenGL''. This has some drawbacks in conjunction with remote visualization: * Rendering takes place on the client, not t..."

The Linux 3D graphics stack is based on ''X11'' and ''OpenGL''. This has some
drawbacks in conjunction with remote visualization:

* Rendering takes place on the client, not the cluster
* Whole 3D model must be transferred via network to the client
* Some OpenGL extensions are not supported when using indirect / client side rendering instead of direct / hardware based rendering
* Many round trips in the X11 protocol negatively influence interactivity
* X11 is not available on non-Linux platforms
* Compatibility problems between client and cluster can occur

To avoid these drawbacks, <code>start_vnc_desktop</code> is provided.
It combines the three open source products [http://www.turbovnc.org/ TurboVNC], [http://www.virtualgl.org/ VirtualGL] and [http://openswr.org/ OpenSWR].

''Virtual Network Computing (VNC)'' is a graphical desktop sharing system.
VNC is platform-independent - there are clients and servers for many
GUI-based operating systems. The VNC server is the program on the
machine that shares its screen. The VNC client (or viewer) is the
program that watches, controls, and interacts with the server. For more
details see: [https://en.wikipedia.org/wiki/VNC Wikipedia]

''VirtualGL'' redirects the 3D rendering commands from Linux OpenGL
applications to 3D accelerator hardware in the cluster. For more details
see: [https://en.wikipedia.org/wiki/VirtualGL Wikipedia]

When no 3D accelerator hardware is available ''OpenSWR'', a high
performance, highly scalable software rasterizer for OpenGL can carry
out the rendering task. For more details see: [http://openswr.org OpenSWR]

This script takes a two step approach to start a VNC server in the
cluster environment:

In the first step the batch system is used to allocate resources where a
VNC server can be started.

In the second step the VNC server is launched on the resources granted
by the batch system. When VNC server is successfully started all
required login credentials and connection parameters will be reported.
To connect to this VNC server a VNC client installation on the local
desktop is required.

= Script usage =

* After login the script can simply be called from the command line:<pre>start_vnc_desktop</pre>
* To get help on the available options use:<pre>start_vnc_desktop --help</pre>
* Hardware rendering is currently only available on FH2 and bwUniCluster, it can be requested with:<pre>start_vnc_desktop --hw-rendering</pre>
* Software rendering is available on all clusters, it can be requested with: <pre>start_vnc_desktop --sw-rendering</pre>
* There is only a limited number of nodes with hardware rendering support, software rendering runs on all nodes.
* For large 3D data sets the software renderer may be faster.
* If neither <code>--hw-rendering</code> nor <code>--sw-rendering</code> is selected no 3D rendering support is available.

= VNC client =

In general every VNC client can be used to connect to the VNC server.
However for best performance and compatibility the use of the
[http://www.turbovnc.org/ TurboVNC] client is recommended.
Below you find the necessary steps for different client operation systems.

; Debian, Ubuntu:
* Download: [https://sourceforge.net/projects/turbovnc/files Download Site] -> latest version -> turbovnc_<VERSION>_amd64.deb
* Install: <pre> sudo apt-get install ./turbovnc_<VERSION>_amd64.deb</pre>
* Execute: <pre>/opt/TurboVNC/bin/vncviewer</pre>

; Red Hat Enterprise Linux, Fedora:
* Download: [https://sourceforge.net/projects/turbovnc/files Download Site] -> latest version -> turbovnc-<VERSION>.x86_64.rpm
* Install: <pre>sudo yum install ./turbovnc-<VERSION>.x86_64.rpm</pre>
* Execute: <pre>/opt/TurboVNC/bin/vncviewer</pre>

; SUSE Linux Enterprise, openSUSE:
* Download [https://sourceforge.net/projects/turbovnc/files Download Site] -> latest version -> turbovnc-<VERSION>.x86_64.rpm
* Install: <pre>sudo zypper install ./turbovnc-<VERSION>.x86_64.rpm</pre>
* Execute: <pre>/opt/TurboVNC/bin/vncviewer</pre>

; ArchLinux:
* Download: Can be installed from the AUR
* Install: <pre>pacaur -S turbovnc</pre>
* Execute: <pre>vncviewer</pre>

; Windows:
* Download: [https://sourceforge.net/projects/turbovnc/files Download Site] -> latest version -> TurboVNC64-<VERSION>.exe for 64-bit, TurboVNC-<VERSION>.exe for 32-bit
* Install: Double click on TurboVNC64-<VERSION>.exe / TurboVNC-<VERSION>.exe. Install in default directory (or chose a different one, if preferred)
* Execute: Java TurboVNCviewer (vncviewer-javaw.bat in installation directory)

[[Category:Visualization]]
[[Category:bwUniCluster]]