NEMO2/SSH-Login-Disabled-Data-Transfer: Difference between revisions
mNo edit summary |
|||
| Line 56: | Line 56: | ||
</pre> |
</pre> |
||
This lists the restricted base directory (<code>/</code>, as set on the command key) |
|||
To copy files: |
|||
itself, so it always works without specifying a path. |
|||
<div style="border: 3px solid #ffc107; background-color: #fff3cd; margin: 10px 0px 10px 0px;"> |
|||
<p style="padding-left: 5px; margin: 0px;">'''Why do I need a full path to copy, but not to view?'''</p> |
|||
The command key restricts <code>rrsync</code> to the base directory <code>/</code> |
|||
(as configured in [[#Step_1:_Create_a_Command_SSH_Key|Step 1]]). Any '''relative''' |
|||
path you pass (e.g. <code>myfiles/</code>) is resolved relative to that base |
|||
directory <code>/</code>, '''not''' your home directory — so |
|||
<code>myfiles/</code> would actually be looked up as <code>/myfiles</code>, which |
|||
does not exist. You must therefore always use the '''full absolute path''', starting |
|||
with <code>/</code>, e.g. your home directory (<code>/home/fr/fr_ab1234</code>). |
|||
</div> |
|||
To copy files, use the full absolute path of your home directory (replace |
|||
<code>/home/fr/fr_ab1234</code> with your own): |
|||
<pre> |
<pre> |
||
rsync -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de: |
rsync -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/home/fr/fr_ab1234/ myfiles/ |
||
</pre> |
</pre> |
||
| Line 76: | Line 91: | ||
rsync -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/work/classic/<USER>-<WS_NAME>/ myfiles/ |
rsync -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/work/classic/<USER>-<WS_NAME>/ myfiles/ |
||
</pre> |
</pre> |
||
== Useful rsync Options == |
|||
For larger transfers, the following options are recommended: |
|||
<pre> |
|||
rsync -ahzP -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/home/fr/fr_ab1234/ myfiles/ |
|||
</pre> |
|||
* <code>-a</code> — archive mode (recursive, preserves permissions/times/symlinks/...) |
|||
* <code>-h</code> — human-readable sizes in the output |
|||
* <code>-z</code> — compress data during transfer (helps on slower connections) |
|||
* <code>-P</code> — show progress and allow resuming partially transferred files (equivalent to <code>--partial --progress</code>, so you don't need to add <code>--progress</code> separately) |
|||
'''Not needed on NEMO2 (WEKA storage):''' <code>-x</code>/<code>--one-file-system</code> |
|||
prevents rsync from crossing into other mounted filesystems while reading the |
|||
source tree. Your NEMO2 home and workspace directories live entirely on the |
|||
WEKA filesystem, so there are no other mounts nested underneath them — this |
|||
option has no effect here and can be omitted. Also make sure not to pass |
|||
<code>-a</code> (or any other option) twice. |
|||
<div style="border: 3px solid #ffc107; background-color: #fff3cd; margin: 10px 0px 10px 0px;"> |
<div style="border: 3px solid #ffc107; background-color: #fff3cd; margin: 10px 0px 10px 0px;"> |
||
Revision as of 18:49, 9 July 2026
Copying Data from NEMO2 While SSH Login is Disabled
While interactive SSH login is temporarily disabled, you can still transfer data
to and from NEMO2 by registering a command SSH key restricted to
rsync. This page walks you through the setup.
Important:
- This only works on login2.
- Your running jobs are not affected and continue to run normally.
Step 1: Create a Command SSH Key
Follow the instructions for registering a Command Key, using
rrsync to restrict the key to read-only access. With
-ro, the key only allows downloading (copying) data from NEMO2 to
your local machine; uploading data to NEMO2 with this key is not possible.
Use the following values when registering the key:
| Field | Value |
|---|---|
| Command | /usr/local/bin/rrsync -ro / |
| From (IP range) | University IP range X.X.0.0/16
|
Note: Replace X.X.0.0/16 with the actual IP range of your own
university/institution.
Tip: You can look up your current IP address at wieistmeineip.scc.kit.edu to determine the correct range for your institution.
University of Freiburg users:
When you visit wieistmeineip.scc.kit.edu, it
shows your external (public) IP address. However, if you reach NEMO2 over an
internal IP address (e.g. a 10.x.x.x network), you must register
that internal/local IP address instead. If you want to use your external IP
address, you need to use the external university network — either via a jump host or
by activating the VPN.
Step 2: Use rsync on login2
Once your command key has been approved, you can use it with rsync
on login2 as follows.
To view the available files:
rsync -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:
This lists the restricted base directory (/, as set on the command key)
itself, so it always works without specifying a path.
Why do I need a full path to copy, but not to view?
The command key restricts rrsync to the base directory /
(as configured in Step 1). Any relative
path you pass (e.g. myfiles/) is resolved relative to that base
directory /, not your home directory — so
myfiles/ would actually be looked up as /myfiles, which
does not exist. You must therefore always use the full absolute path, starting
with /, e.g. your home directory (/home/fr/fr_ab1234).
To copy files, use the full absolute path of your home directory (replace
/home/fr/fr_ab1234 with your own):
rsync -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/home/fr/fr_ab1234/ myfiles/
Replace <USER> with your username and id_ed25519_rrsync
with the private key you registered as the command key.
Workspace example: To view or copy the contents of a workspace, use its full
path (/work/classic/<USER>-<WS_NAME>) instead of myfiles/.
To view the workspace contents:
rsync -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/work/classic/<USER>-<WS_NAME>/
To copy the workspace contents:
rsync -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/work/classic/<USER>-<WS_NAME>/ myfiles/
Useful rsync Options
For larger transfers, the following options are recommended:
rsync -ahzP -e "ssh -i ~/.ssh/id_ed25519_rrsync" <USER>@login2.nemo.uni-freiburg.de:/home/fr/fr_ab1234/ myfiles/
-a— archive mode (recursive, preserves permissions/times/symlinks/...)-h— human-readable sizes in the output-z— compress data during transfer (helps on slower connections)-P— show progress and allow resuming partially transferred files (equivalent to--partial --progress, so you don't need to add--progressseparately)
Not needed on NEMO2 (WEKA storage): -x/--one-file-system
prevents rsync from crossing into other mounted filesystems while reading the
source tree. Your NEMO2 home and workspace directories live entirely on the
WEKA filesystem, so there are no other mounts nested underneath them — this
option has no effect here and can be omitted. Also make sure not to pass
-a (or any other option) twice.
Passphrase-protected key?
If your private key is protected with a passphrase, add it to your local
ssh-agent once before running the commands above, otherwise you
will be prompted for the passphrase on every rsync call:
ssh-add ~/.ssh/id_ed25519_rrsync
Note: You can also configure a matching entry in your ~/.ssh/config
instead of passing -e "ssh -i ..." on every command. Remember that this
only works with login2.