Difference between revisions of "Data Transfer"

From bwHPC Wiki
Jump to: navigation, search
(Blanked the page)
Line 1: Line 1:
  +
== Transfer Tools ==
  +
  +
{|{{Table|width=99%}}
  +
|-
  +
! rowspan="2" | Type
  +
! rowspan="2" | Software
  +
! rowspan="2" | Remarks
  +
! colspan="4" style="text-align:center" | Executable on
  +
! colspan="3" style="text-align:center" | Transfer from/to
  +
|-
  +
!Local°
  +
!bwUniCluster
  +
!bwForCluster
  +
!www
  +
!bwHPC cluster
  +
![[Sds-hd|SDS@hd]]
  +
|-
  +
| rowspan="5" | Command-line
  +
| scp
  +
| rowspan="3" | Throughput < 150 MB/s (depending on cipher)
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
|
  +
| style="text-align:center" | +
  +
| style="text-align:center" |
  +
|-
  +
| sftp
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
|
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
|-
  +
| rsync
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
|
  +
| style="text-align:center" | +
  +
| style="text-align:center" |
  +
|-
  +
| rdata
  +
| Throughput of 350-400 MB/s
  +
|
  +
| style="text-align:center" | +
  +
|
  +
|
  +
|
  +
| style="text-align:center" | +
  +
|-
  +
| wget
  +
| Download from http/ftp address only
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
|
  +
| style="text-align:center" |
  +
|-
  +
| rowspan="2" | Graphical
  +
| [https://winscp.net/eng/download.php WinSCP]
  +
| based on SCP/SFTP, Windows only
  +
| style="text-align:center" | +
  +
|
  +
|
  +
|
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
|-
  +
| [https://filezilla-project.org/download.php?show_all=1 FileZilla]
  +
| based on SFTP
  +
| style="text-align:center" | +
  +
|
  +
|
  +
|
  +
| style="text-align:center" | +
  +
| style="text-align:center" | +
  +
|-
  +
|}
  +
  +
° Depending on the installed operating system (OS).
  +
  +
  +
  +
<H1>Using SFTP from Unix client</H1>
  +
  +
'''Example:'''
  +
  +
<pre>
  +
> sftp ka_xy1234@bwfilestorage.lsdf.kit.edu
  +
Connecting to bwfilestorage.lsdf.kit.edu<br>
  +
ka_xy1234@bwfilestorage.lsdf.kit.edu's password:
  +
sftp> ls
  +
snapshots
  +
temp test
  +
sftp> help
  +
...
  +
sftp> put myfile
  +
sftp> get myfile
  +
</pre>
  +
  +
  +
<H1>Using SFTP from Windows and Mac client</H1>
  +
  +
Windows clients do not have a SCP/SFTP client installed by default, so it needs to be installed before this protocol can be used.
  +
  +
'''Tools for example:'''
  +
* [https://www.openssh.com/ OpenSSH]
  +
*[https://www.chiark.greenend.org.uk/~sgtatham/putty/download.html Putty suite] (for Windows and Unix)
  +
*[https://winscp.net/eng/download.php WinSCP] (for Windows)
  +
*[https://filezilla-project.org/download.php?show_all=1 FileZilla] (for Windows, Mac and Linux)
  +
*[https://cygwin.com/install.html Cygwin] (for Windows)
  +
<br>
  +
'''network drive over SFTP:'''
  +
*[https://www.southrivertechnologies.com/download/downloadwd.html WebDrive] (for Windows and Mac)
  +
*[https://www.eldos.com/sftp-net-drive/comparison.php SFTP Net Drive (ELDOS)] (for Windows)
  +
*[https://www.netdrive.net/ NetDrive] (for Windows)
  +
*[https://www.expandrive.com/expandrive ExpanDrive] (for Windows and Mac)
  +
<hr>
  +
<br>
  +
<br>
  +
<br>
  +
<br>
  +
[[Category:bwFileStorage|SFTP]]

Revision as of 14:37, 14 July 2022

1 Transfer Tools

Type Software Remarks Executable on Transfer from/to
Local° bwUniCluster bwForCluster www bwHPC cluster SDS@hd
Command-line scp Throughput < 150 MB/s (depending on cipher) + + + +
sftp + + + + +
rsync + + + +
rdata Throughput of 350-400 MB/s + +
wget Download from http/ftp address only + + + +
Graphical WinSCP based on SCP/SFTP, Windows only + + +
FileZilla based on SFTP + + +

° Depending on the installed operating system (OS).


2 Using SFTP from Unix client

Example:

> sftp  ka_xy1234@bwfilestorage.lsdf.kit.edu
Connecting to bwfilestorage.lsdf.kit.edu<br>
ka_xy1234@bwfilestorage.lsdf.kit.edu's password: 
sftp> ls
snapshots
temp test
sftp> help
...
sftp> put myfile
sftp> get myfile


3 Using SFTP from Windows and Mac client

Windows clients do not have a SCP/SFTP client installed by default, so it needs to be installed before this protocol can be used.

Tools for example:


network drive over SFTP: