BwUniCluster3.0/Login/Client - Revision history

S Braun: /* Login */

2025-04-02T14:46:27Z

← Older revision		Revision as of 16:46, 2 April 2025
Line 9:		Line 9:

	== Login ==		== Login ==
	To ~~use~~ ~~this~~ ~~client,~~ ~~simply~~ open a command line terminal. Depending on your operating system, the required application is called '''Terminal''', '''Console''', '''PowerShell''', '''Windows-Terminal''' or similar. Inside this terminal, enter the following command to connect to a cluster:		In order to login to bwUniCluster, open a command line terminal. Depending on your operating system, the required application is called '''Terminal''', '''Console''', '''PowerShell''', '''Windows-Terminal''' or similar. Inside this terminal, enter the following command to connect to a cluster:

	<code>		<code>

S Braun: /* Pre-installed SSH Clients */

2025-03-31T07:41:03Z

Pre-installed SSH Clients

← Older revision		Revision as of 09:41, 31 March 2025
Line 8:		Line 8:
	All common operation systems, such as '''Microsoft Windows''', '''Linux''', '''Mac OS''' or other Unix-like operating systems come with a '''built-in SSH client''', most likely provided by the OpenSSH project.		All common operation systems, such as '''Microsoft Windows''', '''Linux''', '''Mac OS''' or other Unix-like operating systems come with a '''built-in SSH client''', most likely provided by the OpenSSH project.

			== Login ==
	To use this client, simply open a command line terminal. Depending on your operating system, the required application is called '''Terminal''', '''Console''', '''PowerShell''', '''Windows-Terminal''' or similar. Inside this terminal, enter the following command to connect to a cluster:		To use this client, simply open a command line terminal. Depending on your operating system, the required application is called '''Terminal''', '''Console''', '''PowerShell''', '''Windows-Terminal''' or similar. Inside this terminal, enter the following command to connect to a cluster:

S Braun at 07:49, 28 March 2025

2025-03-28T07:49:38Z

← Older revision		Revision as of 09:49, 28 March 2025
Line 80:		Line 80:

	For installing WSL and a Linux distribution, pelase refer to [https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux].		For installing WSL and a Linux distribution, pelase refer to [https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux].

	= Data Transfer =

	Further alternatives for file and data exchange are available Here: [[Data_Transfer \| Data Transfer]]


	== WebDAV ==
	WebDAV access to the bwUniCluster-Filesystems

	Access via Webbrowser (read-only)

	Visit uc3-webdav.scc.kit.edu and login with your UC3 username and password (not the HPC service password). Here you can get an overview of your data and download single files. To be able to do more, like uploading new files, or downloading complete folders, a suitable client is needed (rclone, davix).

	Access via command line tools

	The authentication to use command line tools on the bwUniCluster is implemented via the OAuth2-Bearer-Token method.
	The Bearer tokens can be created with the oidc-agent [2]. This is available for Linux, MacOs and Windows [3].
	[2] https://indigo-dc.gitbook.io/oidc-agent/
	[3] https://github.com/indigo-dc/oidc-agent#installation

	Example to create a OAuth2-Bearer-Token for using the bwUniCluster with oidc-gen:

	oidc-gen <your shortname> --client-id unicluster_public --client-secret public --issuer https://bwidm.scc.kit.edu/oidc/realms/bwidm/ --redirect-uri http://localhost:<port-number>/biwdm.scc.kit.edu --scope uc3-webdav

	With the command oidc-token you can show or use the token:

	oidc-token <your shortname>


	=== rclone ===

	Rclone is one of the tools for transferring files from/to the HPC Filesystem via WebDAV. It's detection of unchanged files, parallel transfers and solid retry behaviour make it to a good choice. Rclone can be installed on Linux, MacOS and Windows.
	[1] https://rclone.org

	To configure the rclone remote for the bwUniCluster, you will need to use the following URL, your account and your oidc-token shortname:
	url = https://oauth2.uc3-webdav.scc.kit.edu

	The rclone remote can be configured by command line:
	rclone config create <your uc3-webdav name> webdav url https://oauth2.uc3-webdav.scc.kit.edu vendor other user <account> bearer_token_command "oidc-token <your oidc-token shortname>"

	For the interactive setup process with rclone config see <Data Transfer/WebDAV> and use the bearer_token_command "oidc-token <your oidc-token shortname>" instead of the password.

	The syntax to use Rclone is like this:
	rclone [options] subcommand <parameters> <parameters...>

	Copies from local path to the remote path:
	rclone copy </local/path> <your uc3-webdav name>:/home/<oe>/<group>/<user>/<remote path>

	Copies from remote path to the local path
	rclone copy <your uc3-webdav name>:/home/<oe>/<group>/<user>/<local path>

	For more Details about rclone look at Data Transfer/Rclone or https://rclone.org.


	=== curl ===

	Because WebDAV extends HTTP, any HTTP client can access files in the HPC cluster. Given the full URL to a file, Curl and co can download a file or upload a file.

	File-Download:
	curl -H "Authorization: Bearer `oidc-token <your shortname>`" --output <local filename> https://oauth2.uc3-webdav.scc.kit.edu/home/<oe>/<group>/<user>/<remote filename>

	File-Upload:
	curl -H "Authorization: Bearer `oidc-token <your shortname>`" --silent --upload-file <local filename> https://oauth2.uc3-webdav.scc.kit.edu/home/<oe>/<group>/<user>/<remote filename>


	=== davix ===

	Davix is a set of tools to perform primitive operations via WebDAV protocol. It is shipped with most Linux distributions.

	Some examples to list or transfer files are:

	davix-ls -H "Authorization: Bearer `oidc-token <your shortname>`" https://oauth2.uc3-webdav.scc.kit.edu/home/<oe>/<group>/<user>/my-data-set/

	davix-put -H "Authorization: Bearer `oidc-token <your shortname>`" <local filename> https://oauth2.uc3-webdav.scc.kit.edu/home/<oe>/<group>/<user>/<remote filename>

	davix-get -H "Authorization: Bearer `oidc-token <your shortname>`" https://oauth2.uc3-webdav.scc.kit.edu/home/<oe>/<group>/<user>/<remote filename> <local filename>

S Braun at 07:44, 28 March 2025

2025-03-28T07:44:03Z

← Older revision		Revision as of 09:44, 28 March 2025
Line 25:		Line 25:
	All native SSH clients already come with a remote file copy application: <code>scp</code>.<br>		All native SSH clients already come with a remote file copy application: <code>scp</code>.<br>
	For a more detailed explanation on how to do data transfers with scp, please refer to [[Data_Transfer/SCP]].<br>		For a more detailed explanation on how to do data transfers with scp, please refer to [[Data_Transfer/SCP]].<br>
	Further alternatives for file and data exchange are available Here: [[Data_Transfer \| Data Transfer]]

	== X11 Forwarding ==		== X11 Forwarding ==
Line 84:		Line 83:
	= Data Transfer =		= Data Transfer =

	[[Data_Transfer\| Data Transfer]]		Further alternatives for file and data exchange are available Here: [[Data_Transfer \| Data Transfer]]

S Braun at 07:43, 28 March 2025

2025-03-28T07:43:15Z

← Older revision		Revision as of 09:43, 28 March 2025
Line 24:		Line 24:
	== File Transfer ==		== File Transfer ==
	All native SSH clients already come with a remote file copy application: <code>scp</code>.<br>		All native SSH clients already come with a remote file copy application: <code>scp</code>.<br>
	For a more detailed explanation on how to do data transfers with scp, please refer to [[Data_Transfer/SCP]].		For a more detailed explanation on how to do data transfers with scp, please refer to [[Data_Transfer/SCP]].<br>
			Further alternatives for file and data exchange are available Here: [[Data_Transfer \| Data Transfer]]

	== X11 Forwarding ==		== X11 Forwarding ==

S Braun at 07:41, 28 March 2025

2025-03-28T07:41:28Z

← Older revision		Revision as of 09:41, 28 March 2025
Line 80:		Line 80:

	For installing WSL and a Linux distribution, pelase refer to [https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux].		For installing WSL and a Linux distribution, pelase refer to [https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux].

			= Data Transfer =

			[[Data_Transfer\| Data Transfer]]


			== WebDAV ==
			WebDAV access to the bwUniCluster-Filesystems

			Access via Webbrowser (read-only)

			Visit uc3-webdav.scc.kit.edu and login with your UC3 username and password (not the HPC service password). Here you can get an overview of your data and download single files. To be able to do more, like uploading new files, or downloading complete folders, a suitable client is needed (rclone, davix).

			Access via command line tools

			The authentication to use command line tools on the bwUniCluster is implemented via the OAuth2-Bearer-Token method.
			The Bearer tokens can be created with the oidc-agent [2]. This is available for Linux, MacOs and Windows [3].
			[2] https://indigo-dc.gitbook.io/oidc-agent/
			[3] https://github.com/indigo-dc/oidc-agent#installation

			Example to create a OAuth2-Bearer-Token for using the bwUniCluster with oidc-gen:

			oidc-gen <your shortname> --client-id unicluster_public --client-secret public --issuer https://bwidm.scc.kit.edu/oidc/realms/bwidm/ --redirect-uri http://localhost:<port-number>/biwdm.scc.kit.edu --scope uc3-webdav

			With the command oidc-token you can show or use the token:

			oidc-token <your shortname>


			=== rclone ===

			Rclone is one of the tools for transferring files from/to the HPC Filesystem via WebDAV. It's detection of unchanged files, parallel transfers and solid retry behaviour make it to a good choice. Rclone can be installed on Linux, MacOS and Windows.
			[1] https://rclone.org

			To configure the rclone remote for the bwUniCluster, you will need to use the following URL, your account and your oidc-token shortname:
			url = https://oauth2.uc3-webdav.scc.kit.edu

			The rclone remote can be configured by command line:
			rclone config create <your uc3-webdav name> webdav url https://oauth2.uc3-webdav.scc.kit.edu vendor other user <account> bearer_token_command "oidc-token <your oidc-token shortname>"

			For the interactive setup process with rclone config see <Data Transfer/WebDAV> and use the bearer_token_command "oidc-token <your oidc-token shortname>" instead of the password.

			The syntax to use Rclone is like this:
			rclone [options] subcommand <parameters> <parameters...>

			Copies from local path to the remote path:
			rclone copy </local/path> <your uc3-webdav name>:/home/<oe>/<group>/<user>/<remote path>

			Copies from remote path to the local path
			rclone copy <your uc3-webdav name>:/home/<oe>/<group>/<user>/<local path>

			For more Details about rclone look at Data Transfer/Rclone or https://rclone.org.


			=== curl ===

			Because WebDAV extends HTTP, any HTTP client can access files in the HPC cluster. Given the full URL to a file, Curl and co can download a file or upload a file.

			File-Download:
			curl -H "Authorization: Bearer `oidc-token <your shortname>`" --output <local filename> https://oauth2.uc3-webdav.scc.kit.edu/home/<oe>/<group>/<user>/<remote filename>

			File-Upload:
			curl -H "Authorization: Bearer `oidc-token <your shortname>`" --silent --upload-file <local filename> https://oauth2.uc3-webdav.scc.kit.edu/home/<oe>/<group>/<user>/<remote filename>


			=== davix ===

			Davix is a set of tools to perform primitive operations via WebDAV protocol. It is shipped with most Linux distributions.

			Some examples to list or transfer files are:

			davix-ls -H "Authorization: Bearer `oidc-token <your shortname>`" https://oauth2.uc3-webdav.scc.kit.edu/home/<oe>/<group>/<user>/my-data-set/

			davix-put -H "Authorization: Bearer `oidc-token <your shortname>`" <local filename> https://oauth2.uc3-webdav.scc.kit.edu/home/<oe>/<group>/<user>/<remote filename>

			davix-get -H "Authorization: Bearer `oidc-token <your shortname>`" https://oauth2.uc3-webdav.scc.kit.edu/home/<oe>/<group>/<user>/<remote filename> <local filename>

S Braun: /* Windows Subsystem for Linux */

2025-02-26T16:08:22Z

Windows Subsystem for Linux

← Older revision		Revision as of 18:08, 26 February 2025
Line 77:		Line 77:
	After installing a Linux distribution of choice, you can use the instructions for Linux, e.g. for SSH connections, file transfers, X11 forwarding and other tasks.		After installing a Linux distribution of choice, you can use the instructions for Linux, e.g. for SSH connections, file transfers, X11 forwarding and other tasks.

	WSL is for you if you have to use a Windows computer, e.g. provided by your employer, and if you want to benefit from all the tools and features for code development, debugging and connectivity to HPC systems that Linux ~~computers provide~~.		WSL is for you if you have to use a Windows computer, e.g. provided by your employer, and if you want to benefit from all the tools and features for code development, debugging and connectivity to HPC systems that Linux provides.

	For installing WSL and a Linux distribution, pelase refer to [https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux].		For installing WSL and a Linux distribution, pelase refer to [https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux].

S Braun: /* Windows Subsystem for Linux */

2025-02-26T16:07:17Z

Windows Subsystem for Linux

← Older revision		Revision as of 18:07, 26 February 2025
Line 73:		Line 73:

	== Windows Subsystem for Linux ==		== Windows Subsystem for Linux ==
	Windows Subsystem for Linux (WSL) is not a ~~SSC~~ client but a very tightly integrated, fully-fledged Linux operating system that can be installed and used like a normal Windows application.		Windows Subsystem for Linux (WSL) is not a SSH client but a very tightly integrated, fully-fledged Linux operating system that can be installed and used like a normal Windows application.

	After installing a Linux distribution of choice, you can use the instructions for Linux, e.g. for SSH connections, file transfers, X11 forwarding and other tasks.		After installing a Linux distribution of choice, you can use the instructions for Linux, e.g. for SSH connections, file transfers, X11 forwarding and other tasks.

			WSL is for you if you have to use a Windows computer, e.g. provided by your employer, and if you want to benefit from all the tools and features for code development, debugging and connectivity to HPC systems that Linux computers provide.

	For installing WSL and a Linux distribution, pelase refer to [https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux].		For installing WSL and a Linux distribution, pelase refer to [https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux].

S Braun: /* Windows Subsystem for Linux */

2025-02-26T16:04:04Z

Windows Subsystem for Linux

← Older revision		Revision as of 18:04, 26 February 2025
Line 73:		Line 73:

	== Windows Subsystem for Linux ==		== Windows Subsystem for Linux ==
	Windows Subsystem for Linux is a very tightly integrated, fully-fledged Linux operating system that can be installed and used like a normal Windows application.		Windows Subsystem for Linux (WSL) is not a SSC client but a very tightly integrated, fully-fledged Linux operating system that can be installed and used like a normal Windows application.

	After installing a Linux distribution of choice, you can use the instructions for Linux, e.g. for SSH connections, file transfers, X11 forwarding and other tasks.		After installing a Linux distribution of choice, you can use the instructions for Linux, e.g. for SSH connections, file transfers, X11 forwarding and other tasks.

S Braun: /* Windows Subsystem for Linux */

2025-02-26T16:03:27Z

Windows Subsystem for Linux

← Older revision		Revision as of 18:03, 26 February 2025
Line 73:		Line 73:

	== Windows Subsystem for Linux ==		== Windows Subsystem for Linux ==
			Windows Subsystem for Linux is a very tightly integrated, fully-fledged Linux operating system that can be installed and used like a normal Windows application.

	After installing a Linux ~~system~~ ~~with~~ ~~WSL~~ you can use the instructions for Linux.		After installing a Linux distribution of choice, you can use the instructions for Linux, e.g. for SSH connections, file transfers, X11 forwarding and other tasks.

	[https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux].		For installing WSL and a Linux distribution, pelase refer to [https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux].