SDS@hd/Access: Difference between revisions

From bwHPC Wiki
Jump to navigation Jump to search
(new version fitting to new data transfer page(s))
m (reordered and corrected table, added ssh, switched position of two chapters)
 
Line 6: Line 6:
* You need to be in the belwue-Network. This means you have to use the VPN Service of your HomeOrganization, if you want to access SDS@hd from outside the bwHPC-Clusters (e.g. via eduroam or from your personal notebook).
* You need to be in the belwue-Network. This means you have to use the VPN Service of your HomeOrganization, if you want to access SDS@hd from outside the bwHPC-Clusters (e.g. via eduroam or from your personal notebook).


== Needed information, independent of the chosen tool ==
== Needed Information, independent of the chosen tool ==


* [[Registration/Login/Username| Username]]: Same as for the bwHPC Clusters
* [[Registration/Login/Username| Username]]: Same as for the bwHPC Clusters
Line 16: Line 16:
* Port: 22
* Port: 22


== Recommended way of access ==
== Recommended Setup ==
The following graphic shows the recommended way for accessing SDS@hd via Windows/Mac/Linux. The table provides an overview of the most important access options and links to the related pages.<br />
The following graphic shows the recommended way for accessing SDS@hd via Windows/Mac/Linux. The table provides an overview of the most important access options and links to the related pages.<br />
If you have various use cases, it is recommended to use [[Data_Transfer/Rclone|Rclone]]. You can copy, sync and mount with it and it works not just for SDS@hd but also for the bwHPC clusters.<br />
If you have various use cases, it is recommended to use [[Data_Transfer/Rclone|Rclone]]. You can copy, sync and mount with it and it works not just for SDS@hd but also for the bwHPC clusters.<br />
Line 24: Line 24:
<p style="text-align: center; font-size: small; margin-top: 10px">Figure 1: SDS@hd main transfer routes</p>
<p style="text-align: center; font-size: small; margin-top: 10px">Figure 1: SDS@hd main transfer routes</p>


{| class="wikitable" style="vertical-align:middle;"
{| class="wikitable"
|- style="font-weight:bold;"
|- style="font-weight:bold; text-align:center; vertical-align:middle;"
!
!
! Use Case
! Use Case
Line 34: Line 34:
! Firewall Ports
! Firewall Ports
|-
|-
| [[SDS@hd/Access/SMB|SMB]]
| [[Data_Transfer/Graphical_Clients#MobaXterm|MobaXterm]]
| mount as network drive in file explorer
| Graphical User Interface (GUI)
| [[Data_Transfer/Graphical_Clients#MobaXterm|✓]]
| [[SDS@hd/Access/SMB#Windows|✓]]
| [[SDS@hd/Access/SMB#Mac|✓]]
| ☓
| [[SDS@hd/Access/SMB#Linux|✓]]
| ☓
| up to 40 Gbit/sec
| see sftp
| 139 (netbios), 135 (rpc), 445 (smb)
| see sftp
|- style="vertical-align:middle;"
|-
| [[Data_Transfer/Rclone|Rclone]]
| [[Data_Transfer/Rclone|Rclone]]
| copy, sync and mount with WebDAV
| copy, sync and mount with WebDAV
Line 50: Line 50:
| 80 (http), 443 (https)
| 80 (http), 443 (https)
|-
|-
| [[Data_Transfer/Graphical_Clients#MobaXterm|MobaXterm]]
| [[SDS@hd/Access/SMB|SMB]]
| Graphical User Interface (GUI)
| mount
| [[SDS@hd/Access/SMB#Windows|✓]]
| [[Data_Transfer/Graphical_Clients#MobaXterm|✓]]
| ☓
| [[SDS@hd/Access/SMB#Mac|✓]]
| ☓
| [[SDS@hd/Access/SMB#Linux|✓]]
| see sftp
| up to 40 Gbit/sec
| see sftp
| 139 (netbios), 135 (rpc)
|-
|-
| [[SDS@hd/Access/NFS|NFS]]
| [[SDS@hd/Access/NFS|NFS]]
Line 64: Line 64:
| [[SDS@hd/Access/NFS|✓]]
| [[SDS@hd/Access/NFS|✓]]
| up to 40 Gbit/sec
| up to 40 Gbit/sec
| -
| 445 (smb)
|-
|-
| [[Data_Transfer/SSHFS|SSHFS]]
| mount
| ☓
| [[Data_Transfer/SSHFS#MacOS_&_Linux|✓]]
| [[Data_Transfer/SSHFS#MacOS_&_Linux|✓]]
| see sftp
| see sftp
|- style="vertical-align:middle;"
| [[Data_Transfer/SFTP|SFTP]]
| [[Data_Transfer/SFTP|SFTP]]
| not recommended
| not recommended
Line 76: Line 84:
<p style="text-align: center; font-size: small; margin-top: 10px">Table 1: SDS@hd transfer routes</p>
<p style="text-align: center; font-size: small; margin-top: 10px">Table 1: SDS@hd transfer routes</p>


=== Access via Webbrowser ===
=== Access from a bwHPC Cluster ===

Visit [https://lsdf02-webdav.urz.uni-heidelberg.de/ lsdf02-webdav.urz.uni-heidelberg.de] and login with your SDS@hd username and service password. Here you can get an overview of the data in your &quot;Speichervorhaben&quot; and download single files. To be able to do more, like moving data, uploading new files, or downloading complete folders, a suitable client is needed as described above.

=== Access via a bwHPC Cluster ===


'''bwForCluster Helix:'''<br />
'''bwForCluster Helix:'''<br />
Line 87: Line 91:
'''bwForCluster BinAC'''<br />
'''bwForCluster BinAC'''<br />
You can directly access your storage space on all login nodes.
You can directly access your storage space on all login nodes.

=== Access via Webbrowser (read-only) ===

Visit [https://lsdf02-webdav.urz.uni-heidelberg.de/ lsdf02-webdav.urz.uni-heidelberg.de] and login with your SDS@hd username and service password. Here you can get an overview of the data in your &quot;Speichervorhaben&quot; and download single files. To be able to do more, like moving data, uploading new files, or downloading complete folders, a suitable client is needed as described above.


== Best Practices ==
== Best Practices ==

Latest revision as of 02:20, 3 December 2024

This page provides an overview on how to access data served by SDS@hd. To get an introduction to data transfer in general, see data transfer.

Prerequisites

  • You need to be registered.
  • You need to be in the belwue-Network. This means you have to use the VPN Service of your HomeOrganization, if you want to access SDS@hd from outside the bwHPC-Clusters (e.g. via eduroam or from your personal notebook).

Needed Information, independent of the chosen tool

Recommended Setup

The following graphic shows the recommended way for accessing SDS@hd via Windows/Mac/Linux. The table provides an overview of the most important access options and links to the related pages.
If you have various use cases, it is recommended to use Rclone. You can copy, sync and mount with it and it works not just for SDS@hd but also for the bwHPC clusters.
For an overview of all connection possibilities, please have a look at all data transfer routes.

Data transfer diagram simple.jpg

Figure 1: SDS@hd main transfer routes

Use Case Windows Mac Linux Possible Bandwith Firewall Ports
SMB mount as network drive in file explorer up to 40 Gbit/sec 139 (netbios), 135 (rpc), 445 (smb)
Rclone copy, sync and mount with WebDAV up to 100GBit/sec 80 (http), 443 (https)
MobaXterm Graphical User Interface (GUI) see sftp see sftp
NFS mount for multi-user environments up to 40 Gbit/sec -
SSHFS mount see sftp see sftp
SFTP not recommended up to 40 Gbit/sec 22 (ssh)

Table 1: SDS@hd transfer routes

Access from a bwHPC Cluster

bwForCluster Helix:
You can directly access your storage space under /mnt/sds-hd/ on all login and compute nodes.

bwForCluster BinAC
You can directly access your storage space on all login nodes.

Access via Webbrowser (read-only)

Visit lsdf02-webdav.urz.uni-heidelberg.de and login with your SDS@hd username and service password. Here you can get an overview of the data in your "Speichervorhaben" and download single files. To be able to do more, like moving data, uploading new files, or downloading complete folders, a suitable client is needed as described above.

Best Practices

  • Managing access rights with ACLs
    -> Please set ACLs either via the Windows terminal server or via bwForCluster Helix. ACL changes won't work when used locally on a mounted directory.
  • Multiuser environment -
    -> Use NFS