NEMO2/SSH-Login-Disabled-Data-Transfer

From bwHPC Wiki
< NEMO2
Revision as of 14:02, 11 July 2026 by M Janczyk (talk | contribs) (→‎Step 1: Create a Command SSH Key)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Copying Data from NEMO2 While SSH Login is Disabled

While interactive SSH login is temporarily disabled, you can still transfer data to and from NEMO2 by registering a command SSH key restricted to rsync. This page walks you through the setup.

Important:

  • This only works on login2.
  • Your running jobs are not affected and continue to run normally.

Step 1: Create a Command SSH Key

Follow the instructions for registering a Command Key, using rrsync to restrict the key to read-only access. With -ro, the key only allows downloading (copying) data from NEMO2 to your local machine; uploading data to NEMO2 with this key is not possible.

Use the following values when registering the key:

Field Value
Command
/usr/local/bin/rrsync -ro /
From (IP range) University IP range X.X.0.0/16

Note: Replace X.X.0.0/16 with the actual IP range of your own university/institution. If you need to allow more than one range (e.g. an internal and an external one), you can specify multiple IP addresses/ranges at once, separated by a comma, e.g. 10.x.x.x/y,132.230.0.0/16.

Tip: You can look up your current IP address at wieistmeineip.scc.kit.edu to determine the correct range for your institution.

University of Freiburg users:

wieistmeineip.scc.kit.edu shows your external IP. Inside the university network you usually have an internal 10.x.x.x address instead — register that one, or use a jump host (which has its own external-facing IP) or VPN. If you might use both, register both ranges separated by a comma.

Step 2: Use rsync on login2

Once your command key has been approved, you can use it with rsync on login2 as follows.

To view the available files:

rsync -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:

This lists the restricted base directory (/, as set on the command key) itself, so it always works without specifying a path.

Why do I need a full path to copy, but not to view?

The command key restricts rrsync to the base directory / (as configured in Step 1). Any relative path you pass (e.g. myfiles/) is resolved relative to that base directory /, not your home directory — so myfiles/ would actually be looked up as /myfiles, which does not exist. You must therefore always use the full absolute path, starting with /, e.g. your home directory (/home/fr/fr_ab1234).

To copy files, use the full absolute path of your home directory (replace /home/fr/fr_ab1234 with your own):

rsync -ahzP -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/home/fr/fr_ab1234/ myfiles/

Replace <USER> with your username and id_ed25519_rrsync with the private key you registered as the command key. The -a option is required to copy folders recursively — without it, rsync only copies top-level files and skips directories. See Useful rsync Options below for what the other options do.

Workspace example: To view or copy the contents of a workspace, use its full path (/work/classic/<USER>-<WS_NAME>) instead of myfiles/.

To view the workspace contents:

rsync -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/work/classic/<USER>-<WS_NAME>/

To copy the workspace contents:

rsync -ahzP -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/work/classic/<USER>-<WS_NAME>/ myfiles/

Useful rsync Options

The copy commands above already use these recommended options:

rsync -ahzP -e "ssh -i ~/.ssh/id_ed25519_rrsync" ...
  • -a — archive mode (recursive, preserves permissions/times/symlinks/...)
  • -h — human-readable sizes in the output
  • -z — compress data during transfer (helps on slower connections)
  • -P — show progress and allow resuming partially transferred files (equivalent to --partial --progress, so you don't need to add --progress separately)

Passphrase-protected key?

If your private key is protected with a passphrase, add it to your local ssh-agent once before running the commands above, otherwise you will be prompted for the passphrase on every rsync call:

ssh-add ~/.ssh/id_ed25519_rrsync

Note: You can also configure a matching entry in your ~/.ssh/config instead of passing -e "ssh -i ..." on every command. Remember that this only works with login2.