Data Transfer/Rclone - Revision history

H Schumacher: Added instructions for adding rclone to path on windows.

2026-06-02T15:30:40Z

Added instructions for adding rclone to path on windows.

@@ Line 17: / Line 17: @@
 * Installation on [https://rclone.org/install/#windows Windows]
 * Installation on [https://rclone.org/install/#macos macOS]
 ** If Rclone shall be used together with sftp, the [[Data_Transfer/SSHFS#Installation | sshfs installation]] instructions must be followed after installing Rclone.
@@ Line 198: / Line 199: @@
 --stats-log-level=DEBUG
 </pre>

H Schumacher: corrected information

2026-05-31T13:42:21Z

corrected information

← Older revision		Revision as of 15:42, 31 May 2026
Line 1:		Line 1:

	[https://rclone.org/docs/ Rclone] is a command line tool to manage files on remote systems (e.g. ~~cloud~~ ~~storage~~ ~~systems).~~ ~~Rclone~~ ~~either~~ ~~synchronizes~~ in ~~one~~ ~~direction~~ ~~only or its mounting functionality is used with~~ ~~<code>rclone mount~~</~~code~~>~~. Data can be piped between two completely remote locations, sometimes without local download. One advantage is that the transfer is multithreaded and it operates on a file level basis.~~		[https://rclone.org/docs/ Rclone] is a command line tool to manage files on remote systems (e.g. high performance clusters, data and cloud storage systems, archiving services). </br>
			Rclone either synchronizes in one direction only or its mounting functionality is used with <code>rclone mount</code>. </br>
	'''Caution:''' You can't use Rclone with 2FA.
			Data can be piped between two completely remote locations. When moving data within the same system and the data transfer protocol allows for server side copy, no local download is necessary. </br>
			One advantage is that the transfer is multithreaded and it operates on a file level basis.</br>
			'''Caution:''' When using Rclone with 2FA it is necessary to connect via ssh key instead of a password.

	== Installation ==		== Installation ==
Line 8:		Line 11:

	# Download the relevant binary.		# Download the relevant binary.
	# Extract the <code>rclone</code> executable, <code>rclone.exe</code> ~~on Windows, from the archive~~.		# Extract the <code>rclone</code> executable (<code>rclone.exe</code>).
	# You can use the executables without further installation. For easy use, it is recommended to add the binary to your PATH environment variable. Information on how to do this can be found below.		# You can use the executables without further installation. For easy use, it is recommended to add the binary to your PATH environment variable. Information on how to do this can be found below.

H Schumacher: improved Windows instructions

2026-05-27T16:01:30Z

improved Windows instructions

← Older revision		Revision as of 18:01, 27 May 2026
Line 135:		Line 135:
	=== Windows ===		=== Windows ===

	To run rclone mount on Windows, you will need to [https://winfsp.dev/rel/ download and install WinFsp]. ~~To mount on drive letter X or a nonexistent subdirectory, use:~~		To run rclone mount on Windows, you will need to [https://winfsp.dev/rel/ download and install WinFsp] first.

			<syntaxhighlight lang="python"># mount under drive letter X (or any other free letter)
	<pre>rclone mount <remote-name>:path/to/files X:
	rclone mount ~~<~~remote-name~~>~~:path/to/files C:~~\path\parent\mount</pre>~~		rclone mount --vfs-cache-mode full <remote-name>:<path/to/remote/files> X:
			# or provide the path to a nonexisting folder:
			rclone mount --vfs-cache-mode full <remote-name>:<path/to/remote/files> <C:\path\parentDir\nonexistingFoldername></syntaxhighlight>
			To unmount, the command can be stopped in the terminal with the key combination CTRL+C. Alternatively, a mounted drive can be unmounted by making a right click on the drive name in the file explorer and choose "Eject". </br>
	In contrast to Linux/Mac, there is no background mode.		In contrast to Linux/Mac, there is no background mode.

Line 147:		Line 150:
	Create an empty directory on your local machine and then execute		Create an empty directory on your local machine and then execute

	<~~pre~~># to mount the root folder:		<syntaxhighlight lang="bash"># to mount the root folder:
	rclone mount --vfs-cache-mode full ~~<~~remote-name~~>~~: /path/to/empty/folder		rclone mount --vfs-cache-mode full <remote-name>: /path/to/empty/folder
	# to mount a subfolder:		# to mount a subfolder:
	rclone mount --vfs-cache-mode full ~~<~~remote-name~~>~~:folderX/folderY /path/to/empty/folder		rclone mount --vfs-cache-mode full <remote-name>:folderX/folderY /path/to/empty/folder
	# to unmount:		# to unmount:
	fusermount -uz /path/to/mounted/folder </~~pre~~>		fusermount -uz /path/to/mounted/folder </syntaxhighlight>

	== Best Practices ==		== Best Practices ==

H Schumacher: fixed typo

2026-05-27T09:44:49Z

fixed typo

← Older revision		Revision as of 11:44, 27 May 2026
Line 15:		Line 15:
	* Installation on [https://rclone.org/install/#windows Windows]		* Installation on [https://rclone.org/install/#windows Windows]
	* Installation on [https://rclone.org/install/#macos macOS]		* Installation on [https://rclone.org/install/#macos macOS]
	** If Rclone shall be used together with sftp, the [[~~https://wiki.bwhpc.de/e/~~Data_Transfer/SSHFS#Installation \| sshfs installation]] instructions must be followed after installing Rclone.		** If Rclone shall be used together with sftp, the [[Data_Transfer/SSHFS#Installation \| sshfs installation]] instructions must be followed after installing Rclone.
	* Installation on [https://rclone.org/install/#script-installation Linux]		* Installation on [https://rclone.org/install/#script-installation Linux]

H Schumacher: changed lsd to lsf

2026-05-18T16:11:02Z

changed lsd to lsf

← Older revision		Revision as of 18:11, 18 May 2026
Line 109:		Line 109:
	<pre>rclone [options] subcommand <parameters> <parameters...>		<pre>rclone [options] subcommand <parameters> <parameters...>
	</pre>		</pre>
	List all ~~directories/containers/buckets~~ in the folder XX.		List all elements in the folder XX:
		⚫	<syntaxhighlight lang="bash">rclone lsf <remote-name>:XX

			# List elements from subfolders as well:
⚫	<~~pre~~>rclone ~~lsd~~ <remote-name>:XX
			rclone lsf --max-depth 2 <remote-name>:XX
	</pre>
			</syntaxhighlight>
	Copies /local/path to the remote path		Copies /local/path to the remote path

H Schumacher: Reordered remote setup options

2026-05-13T14:49:46Z

Reordered remote setup options

← Older revision		Revision as of 16:49, 13 May 2026
Line 15:		Line 15:
	* Installation on [https://rclone.org/install/#windows Windows]		* Installation on [https://rclone.org/install/#windows Windows]
	* Installation on [https://rclone.org/install/#macos macOS]		* Installation on [https://rclone.org/install/#macos macOS]
			** If Rclone shall be used together with sftp, the [[https://wiki.bwhpc.de/e/Data_Transfer/SSHFS#Installation \| sshfs installation]] instructions must be followed after installing Rclone.
	* Installation on [https://rclone.org/install/#script-installation Linux]		* Installation on [https://rclone.org/install/#script-installation Linux]

Line 36:		Line 37:
	* <code>remote-name</code> (for example you can use the name of the service you want to connect to)		* <code>remote-name</code> (for example you can use the name of the service you want to connect to)

	You have three different options to set up a new remote ~~which~~ ~~are~~ ~~explained by~~ the following sections.		You have three different options to set up a new remote as described in the following sections. It's enough to choose one of those.

⚫	<span style="font-size: 1.1em; text-decoration: underline;">'''Interactive Setup''' </span>

⚫	Execute:
	<pre>rclone config</pre>
⚫	This will guide you through an interactive setup process. You can find detailed instructions at the website:

⚫	* [https://rclone.org/webdav/ Connect via webdav]
⚫	* [https://rclone.org/smb/ Connect via smb]
⚫	* [https://rclone.org/sftp/ Connect via sftp]

	<span style="font-size: 1.1em; text-decoration: underline;">'''Oneliner''' </span>		<span style="font-size: 1.1em; text-decoration: underline;">'''Oneliner''' </span>

			Open a terminal and define all parameters in one command. If a ssh key shall be used instead of a password, it might be easier to adjust the config file directly as described in the next section.
	Define all parameters in one command. For example:
	<syntaxhighlight lang="bash"># smb		<syntaxhighlight lang="bash"># smb
	rclone config create <remote-name> smb host=<remotehost> user=<username> domain=<domain> pass=<password> --obscure		rclone config create <remote-name> smb host=<remotehost> user=<username> domain=<domain> pass=<password> --obscure
Line 66:		Line 57:
	rclone config file		rclone config file
	</syntaxhighlight>		</syntaxhighlight>
	~~You~~ ~~can~~ use the following ~~snippet~~ as template for your ~~connections.~~		Open the file and use one of the following snippets as template for your connection:
	<syntaxhighlight lang="bash">[<remote-name>]		<syntaxhighlight lang="bash">[<remote-name>]
	type = smb		type = smb
Line 85:		Line 76:
	key_use_agent = false		key_use_agent = false

	[<remote-name-ssh-version>]		[<remote-name-ssh-key-version>]
	type = sftp		type = sftp
	host = <hostname>		host = <hostname>
Line 96:		Line 87:
	To add the password, please use the following command. Make sure to keep the prepended space so that the command is not saved in your command history.		To add the password, please use the following command. Make sure to keep the prepended space so that the command is not saved in your command history.
	<syntaxhighlight> rclone config update <remote-name> pass=<password> --obscure</syntaxhighlight>		<syntaxhighlight> rclone config update <remote-name> pass=<password> --obscure</syntaxhighlight>

		⚫	<span style="font-size: 1.1em; text-decoration: underline;">'''Interactive Setup''' </span>

		⚫	Execute:
			<syntaxhighlight>rclone config</syntaxhighlight>
		⚫	This will guide you through an interactive setup process. It's quite helpful for checking out which options there are and what the config file looks like after setting values that deviate from the default values. You can find detailed instructions at the website:

		⚫	* [https://rclone.org/webdav/ Connect via webdav]
		⚫	* [https://rclone.org/smb/ Connect via smb]
		⚫	* [https://rclone.org/sftp/ Connect via sftp]

	=== Use Remote ===		=== Use Remote ===

			To see all configured remotes:
			<syntaxhighlight lang="bash">rclone config
			# Exit with 'q'.
			</syntaxhighlight>

	The syntax to use Rclone is like this:		The syntax to use Rclone is like this:
Line 123:		Line 129:
	== Usage Rclone Mount ==		== Usage Rclone Mount ==

	Before you can follow the instructions in this chapter, you need to have set up a [[Data_Transfer/Rclone#Usage_Rclone \| remote]].		Before you can follow the instructions in this chapter, you need to have set up a [[Data_Transfer/Rclone#Usage_Rclone \| remote]].</br>
	~~Detailed~~ information on how to use rclone mount can be found [https://rclone.org/commands/rclone_mount/ here].		More detailed information on how to use rclone mount can be found [https://rclone.org/commands/rclone_mount/ here].

	=== Windows ===		=== Windows ===

H Schumacher: Added smb to oneliner and config file

2026-05-06T11:26:59Z

Added smb to oneliner and config file

← Older revision		Revision as of 13:26, 6 May 2026
Line 51:		Line 51:

	Define all parameters in one command. For example:		Define all parameters in one command. For example:
	<syntaxhighlight lang="bash"># ~~sftp~~		<syntaxhighlight lang="bash"># smb
			rclone config create <remote-name> smb host=<remotehost> user=<username> domain=<domain> pass=<password> --obscure
			# sftp
	rclone config create <remote-name> sftp host=<remotehost> user=<username> pass=<password> --obscure		rclone config create <remote-name> sftp host=<remotehost> user=<username> pass=<password> --obscure
	# webdav		# webdav
Line 58:		Line 60:

	<span style="font-size: 1.1em; text-decoration: underline;">'''Adjust Config File''' </span>		<span style="font-size: 1.1em; text-decoration: underline;">'''Adjust Config File''' </span>
			<syntaxhighlight lang="bash">

	To ~~see,~~ ~~where~~ ~~the~~ file is ~~run:~~ ~~<code>rclone~~ config file~~</code>~~.		# Make sure that a config file exists. Access rights for the config file are set to 600.
			rclone config touch

			# Get the location of the file:
			rclone config file
			</syntaxhighlight>
	You can use the following snippet as template for your connections.		You can use the following snippet as template for your connections.
	<syntaxhighlight>[<remote-name>]		<syntaxhighlight lang="bash">[<remote-name>]
			type = smb
			host = <hostname>
			user = <userID>
			domain = <e.g. BWSERVICESAD>

			[<remote-name>]
	type = webdav		type = webdav
	url = <hostURL>		url = <hostURL>
Line 73:		Line 84:
	user = <userID>		user = <userID>
	key_use_agent = false		key_use_agent = false

			[<remote-name-ssh-version>]
			type = sftp
			host = <hostname>
			user = <userID>
			key_file = ~/.ssh/id_ed25519
			pubkey_file = ~/.ssh/id_ed25519.pub
			key_use_agent = true
			shell_type = none
	</syntaxhighlight>		</syntaxhighlight>
	To add the password, please use		To add the password, please use the following command. Make sure to keep the prepended space so that the command is not saved in your command history.
	<syntaxhighlight>rclone config update <remote-name> pass=<password> --obscure</syntaxhighlight>		<syntaxhighlight> rclone config update <remote-name> pass=<password> --obscure</syntaxhighlight>

	=== Use Remote ===		=== Use Remote ===

H Schumacher: added oneliner for webdav

2026-02-16T16:12:21Z

added oneliner for webdav

← Older revision		Revision as of 18:12, 16 February 2026
Line 51:		Line 51:

	Define all parameters in one command. For example:		Define all parameters in one command. For example:
	<syntaxhighlight lang="bash">~~rclone config create test~~ sftp ~~host=<remotehost> user=<username> pass=<password> --obscure</syntaxhighlight>~~		<syntaxhighlight lang="bash"># sftp
			rclone config create <remote-name> sftp host=<remotehost> user=<username> pass=<password> --obscure

			# webdav
			rclone config create <remote-name> webdav url=<remotehost> user=<username> pass=<password> --obscure
			</syntaxhighlight>

	<span style="font-size: 1.1em; text-decoration: underline;">'''Adjust Config File''' </span>		<span style="font-size: 1.1em; text-decoration: underline;">'''Adjust Config File''' </span>

H Schumacher: Fix wrong link naming

2025-10-23T08:21:16Z

Fix wrong link naming

← Older revision		Revision as of 10:21, 23 October 2025
Line 44:		Line 44:
	This will guide you through an interactive setup process. You can find detailed instructions at the website:		This will guide you through an interactive setup process. You can find detailed instructions at the website:

	* [https://rclone.org/webdav/ Connect via ~~sftp~~]		* [https://rclone.org/webdav/ Connect via webdav]
	* [https://rclone.org/smb/ Connect via smb]		* [https://rclone.org/smb/ Connect via smb]
	* [https://rclone.org/sftp/ Connect via ~~webdav~~]		* [https://rclone.org/sftp/ Connect via sftp]

	<span style="font-size: 1.1em; text-decoration: underline;">'''Oneliner''' </span>		<span style="font-size: 1.1em; text-decoration: underline;">'''Oneliner''' </span>

H Schumacher: Added unmount instruction + alternative to daemon flag

2025-03-20T16:42:15Z

Added unmount instruction + alternative to daemon flag

← Older revision		Revision as of 18:42, 20 March 2025
Line 112:		Line 112:
	=== MacOS & Linux ===		=== MacOS & Linux ===

	You can run mount in either foreground or background (aka daemon) mode. Mount runs in foreground mode by default. Use the <code>--daemon</code> flag to force background mode.		You can run mount in either foreground or background (aka daemon) mode. Mount runs in foreground mode by default. Use the <code>--daemon</code> flag to force background mode. If this doesn't work, you can put an <code>&</code> at the end of the command instead.

	Create an empty directory on your local machine and then execute		Create an empty directory on your local machine and then execute
Line 119:		Line 119:
	rclone mount --vfs-cache-mode full <remote-name>: /path/to/empty/folder		rclone mount --vfs-cache-mode full <remote-name>: /path/to/empty/folder
	# to mount a subfolder:		# to mount a subfolder:
	rclone mount --vfs-cache-mode full <remote-name>:folderX/folderY /path/to/empty/folder ~~</pre>~~		rclone mount --vfs-cache-mode full <remote-name>:folderX/folderY /path/to/empty/folder
			# to unmount:
			fusermount -uz /path/to/mounted/folder </pre>

	== Best Practices ==		== Best Practices ==