bwHPC Wiki - User contributions [en]

SDS@hd/Access/NFS

2025-08-08T07:55:09Z

S Siebler:

NFS is a Network File System only for Linux.

== Prerequisites ==

* The access via nfs protocol is machine-based, which means new nfs-Clients have to be registered on SDS@hd. During this registration each machine receives a keytab file from the SDS@hd team, which allows mounting SDS@hd. In order the keytab file to be created, please send an email to SDS@hd team for Clientregistration with the following information:
** hostname of the new nfs-Client
** IP address
** short description
** location
** acronym of the Speichervorhaben which should be available on this machine

== Using NFSv4 for UNIX client ==

The authentication for data access via NFSv4 is performed using Kerberostickets. This requires a functioning Kerberos environment on the client!

{{:SDS@hd/Access/Kerberos}}

After configuring kerberos, you have to install nfs packages in your system, and enable kerberized NFSv4. The exact names of the packages depending on you linux distribution (see examples below).

''Example RedHat/CentOS''
<pre>
> yum install nfs-utils nfs4-acl-tools

/etc/sysconfig/nfs:
NEED_IDMAPD=yes
NEED_GSSD=yes
</pre>

''Example Debian/Ubuntu''
<pre>
> apt install nfs-common nfs4-acl-tools nfs-server

/etc/default/nfs-common:
NEED_IDMAPD=yes
NEED_GSSD=yes
</pre>
On ubuntu server: nfs-kernel-server

{{:SDS@hd/Access/ID-Mapping}}

To enable the ID-Mapping for NFSv4 mounts change the file ''/etc/idmapd.conf'' with the following lines:
<pre>
in /etc/idmapd.conf:
[General]
Domain = urz.uni-heidelberg.de
Local-Realms = BWSERVICES.UNI-HEIDELBERG.DE
</pre>

=== Mount a nfs share ===
The usual restrictions for mounting drives under Linux apply. Usually this can only be done by the superuser "root". For detailed information, please contact the system administrator of your system.

After successfull configuration (s. 2.1) you can mount your SDS@hd share with the following commands:
<pre>
> mkdir <mountpoint>
> mount -t nfs4 -o sec=krb5,vers=4 lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02/ <mountpoint>
</pre>

To enable the mounting after a restart, you have to add the following line to the file "/etc/fstab"
<pre>
lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02/ <mountpoint> nfs4 sec=krb5,vers=4 0 0
</pre>

==== AutoFS Setup ====

Instead of the fstab-entry you can also use the automounter "autofs".

* RedHat/CentOS:
<pre>
$ yum install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

* Debian/Ubuntu:
<pre>
$ apt install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

Afterwards you configure the SDS@hd Speichervorhaben in a new map file:
<pre>
$ cat /etc/auto.sds-hd
sds-hd -fstype=nfs4,rw,sec=krb5,vers=4,nosuid,nodev lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02
....
</pre>

You have to include the new map into the auto.master file, e.g.:
<pre>
$ cat /etc/auto.master
[...]
/mnt /etc/auto.sds-hd
[...]
</pre>

To display all available SDS@hd shares on this machine to the users, you should enable "browser_mode":
<pre>
$ cat /etc/autofs.conf
[...]
# to display all available SDS-hd shares on this to the users
browse_mode=yes
[...]
</pre>
otherwise each share-folder will only be visible after a user has mounted.

After changing the configuration, you should restart the autofs daemon, e.g.:
<pre>
$ systemctl restart autofs
</pre>

Of course you can adopt all other autofs options, like timeouts, etc. to the specific needs of your environment or use any other method for dynamically mounting the shares.

=== Access your data ===
'''Attention!''' The access can not be done as root user, because root uses the Kerberosticket of the machine, which does not have data access!

To access your data on SDS@hd you have to fetch a valid kerberos ticket with your SDS@hd user and Servicepassword:
<pre>
> kinit hd_xy123
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
</pre>
You can check afterwards your kerberos ticket with:
<pre>
> klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE

Valid starting Expires Service principal
20.09.2017 04:00:01 21.09.2017 04:00:01 krbtgt/BWSERVICES.UNI-HEIDELBERG.DE@BWSERVICES.UNI-HEIDELBERG.DE
renew until 29.09.2017 13:38:49
</pre>

Afterwards you should be able to access the mountpoint, which contain all Speichervorhaben exported to your machine:
<pre>
> ls <mountpoint>
sd16j007 sd17c010 sd17d005
</pre>

=== Renew a kerberos ticket ===
Because a kerberos ticket has a limited lifetime (default: 10 hours, maximum 24 hours) for security reasons, you have to renew your ticket before it expires to prevent access loss.
<pre>
> kinit -R
</pre>

This renewal could only be done for maximum time of 10 Days and as long as the current kerberos ticket is still valid. For renewal of an expired ticket, you have to use again your Servicepassword.

=== Destroy kerberos ticket ===
Even if kerberos tickets are only valid for a limited period of time, a ticket should be destroyed as soon as access is no longer needed to prevent misuse on multi-user systems:
<pre>kdestroy</pre>

=== Automated kerberos tickets ===
<span style="color:red"><strong>'''Attention!''' Keep this generated Keytab safe and use it only in trusted environments!</strong></span>

If your workflow needs a permanent access to SDS@hd for longer than 10 Days, you can use '''ktutil''' to encrypt your Service Password into a keytab file:

''Interactive way:''
<pre>
ktutil
ktutil: addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e rc4-hmac
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
ktutil: addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e aes256-cts
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
ktutil: wkt xy123.keytab
ktuitl: quit
</pre>

''Non-interactive way:''
<pre>
echo -e "addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e rc4-hmac\n<your_servicepasword>\n
addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e aes256-cts\n<your_servicepasword>\nwkt xy123.keytab" | ktutil
</pre>

With this keytab, you can fetch a kerberos ticket without an interactive password:
<pre>
kinit -k -t xy123.keytab hd_xy123
</pre>
<br>

== Troubleshooting ==

=== Incorrect mount option ===

When you try to mount SDS@hd you get the following error message:
<pre>
mount.nfs4: mount(2): Invalid argument
mount.nfs4: an incorrect mount option was specified
</pre>

This means the service rpc-gssd.service is not yet running. Please try to start the service via <pre>systemctl restart rpc-gssd.service</pre>

=== Permission denied while mounting when SELinux is enabled ===

When trying to mount, you get the following error message:
<pre>
mount.nfs4: mount(2): Permission denied
mount.nfs4: trying text-based options 'sec=krb5,vers=4,addr=x.x.x.x,clientaddr=a.b.c.d'
mount.nfs4: mount(2): Permission denied
mount.nfs4: access denied by server while mounting lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02
</pre>

You should check your systemlogs for entries like:
<pre>
Aug 07 13:09:53 systemname setroubleshoot[510523]: SELinux is preventing /usr/sbin/rpc.gssd from open access on the file /etc/krb5.keytab. For complete SELinux messages run: sealert -l XXXXXX
</pre>

or similar. To solve the issue you have to add a SELinux rule to allow access to /etc/krb5.keytab (or disable SElinux completely).

SDS@hd/Access/NFS

2025-08-08T07:54:33Z

S Siebler: /* Mount a nfs share */

NFS is a Network File System only for Linux.

== Prerequisites ==

* The access via nfs protocol is machine-based, which means new nfs-Clients have to be registered on SDS@hd. During this registration each machine receives a keytab file from the SDS@hd team, which allows mounting SDS@hd. In order the keytab file to be created, please send an email to SDS@hd team for Clientregistration with the following information:
** hostname of the new nfs-Client
** IP address
** short description
** location
** acronym of the Speichervorhaben which should be available on this machine

== Using NFSv4 for UNIX client ==

The authentication for data access via NFSv4 is performed using Kerberostickets. This requires a functioning Kerberos environment on the client!

{{:SDS@hd/Access/Kerberos}}

After configuring kerberos, you have to install nfs packages in your system, and enable kerberized NFSv4. The exact names of the packages depending on you linux distribution (see examples below).

''Example RedHat/CentOS''
<pre>
> yum install nfs-utils nfs4-acl-tools

/etc/sysconfig/nfs:
NEED_IDMAPD=yes
NEED_GSSD=yes
</pre>

''Example Debian/Ubuntu''
<pre>
> apt install nfs-common nfs4-acl-tools nfs-server

/etc/default/nfs-common:
NEED_IDMAPD=yes
NEED_GSSD=yes
</pre>
On ubuntu server: nfs-kernel-server

{{:SDS@hd/Access/ID-Mapping}}

To enable the ID-Mapping for NFSv4 mounts change the file ''/etc/idmapd.conf'' with the following lines:
<pre>
in /etc/idmapd.conf:
[General]
Domain = urz.uni-heidelberg.de
Local-Realms = BWSERVICES.UNI-HEIDELBERG.DE
</pre>

=== Mount a nfs share ===
The usual restrictions for mounting drives under Linux apply. Usually this can only be done by the superuser "root". For detailed information, please contact the system administrator of your system.

After successfull configuration (s. 2.1) you can mount your SDS@hd share with the following commands:
<pre>
> mkdir <mountpoint>
> mount -t nfs4 -o sec=krb5,vers=4 lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02/ <mountpoint>
</pre>

To enable the mounting after a restart, you have to add the following line to the file "/etc/fstab"
<pre>
lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02/ <mountpoint> nfs4 sec=krb5,vers=4 0 0
</pre>

==== AutoFS Setup ====

Instead of the fstab-entry you can also use the automounter "autofs".

* RedHat/CentOS:
<pre>
$ yum install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

* Debian/Ubuntu:
<pre>
$ apt install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

Afterwards you configure the SDS@hd Speichervorhaben in a new map file:
<pre>
$ cat /etc/auto.sds-hd
sds-hd -fstype=nfs4,rw,sec=krb5,vers=4.1,nosuid,nodev lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02
....
</pre>

You have to include the new map into the auto.master file, e.g.:
<pre>
$ cat /etc/auto.master
[...]
/mnt /etc/auto.sds-hd
[...]
</pre>

To display all available SDS@hd shares on this machine to the users, you should enable "browser_mode":
<pre>
$ cat /etc/autofs.conf
[...]
# to display all available SDS-hd shares on this to the users
browse_mode=yes
[...]
</pre>
otherwise each share-folder will only be visible after a user has mounted.

After changing the configuration, you should restart the autofs daemon, e.g.:
<pre>
$ systemctl restart autofs
</pre>

Of course you can adopt all other autofs options, like timeouts, etc. to the specific needs of your environment or use any other method for dynamically mounting the shares.

=== Access your data ===
'''Attention!''' The access can not be done as root user, because root uses the Kerberosticket of the machine, which does not have data access!

To access your data on SDS@hd you have to fetch a valid kerberos ticket with your SDS@hd user and Servicepassword:
<pre>
> kinit hd_xy123
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
</pre>
You can check afterwards your kerberos ticket with:
<pre>
> klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE

Valid starting Expires Service principal
20.09.2017 04:00:01 21.09.2017 04:00:01 krbtgt/BWSERVICES.UNI-HEIDELBERG.DE@BWSERVICES.UNI-HEIDELBERG.DE
renew until 29.09.2017 13:38:49
</pre>

Afterwards you should be able to access the mountpoint, which contain all Speichervorhaben exported to your machine:
<pre>
> ls <mountpoint>
sd16j007 sd17c010 sd17d005
</pre>

=== Renew a kerberos ticket ===
Because a kerberos ticket has a limited lifetime (default: 10 hours, maximum 24 hours) for security reasons, you have to renew your ticket before it expires to prevent access loss.
<pre>
> kinit -R
</pre>

This renewal could only be done for maximum time of 10 Days and as long as the current kerberos ticket is still valid. For renewal of an expired ticket, you have to use again your Servicepassword.

=== Destroy kerberos ticket ===
Even if kerberos tickets are only valid for a limited period of time, a ticket should be destroyed as soon as access is no longer needed to prevent misuse on multi-user systems:
<pre>kdestroy</pre>

=== Automated kerberos tickets ===
<span style="color:red"><strong>'''Attention!''' Keep this generated Keytab safe and use it only in trusted environments!</strong></span>

If your workflow needs a permanent access to SDS@hd for longer than 10 Days, you can use '''ktutil''' to encrypt your Service Password into a keytab file:

''Interactive way:''
<pre>
ktutil
ktutil: addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e rc4-hmac
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
ktutil: addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e aes256-cts
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
ktutil: wkt xy123.keytab
ktuitl: quit
</pre>

''Non-interactive way:''
<pre>
echo -e "addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e rc4-hmac\n<your_servicepasword>\n
addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e aes256-cts\n<your_servicepasword>\nwkt xy123.keytab" | ktutil
</pre>

With this keytab, you can fetch a kerberos ticket without an interactive password:
<pre>
kinit -k -t xy123.keytab hd_xy123
</pre>
<br>

== Troubleshooting ==

=== Incorrect mount option ===

When you try to mount SDS@hd you get the following error message:
<pre>
mount.nfs4: mount(2): Invalid argument
mount.nfs4: an incorrect mount option was specified
</pre>

This means the service rpc-gssd.service is not yet running. Please try to start the service via <pre>systemctl restart rpc-gssd.service</pre>

=== Permission denied while mounting when SELinux is enabled ===

When trying to mount, you get the following error message:
<pre>
mount.nfs4: mount(2): Permission denied
mount.nfs4: trying text-based options 'sec=krb5,vers=4.1,addr=x.x.x.x,clientaddr=a.b.c.d'
mount.nfs4: mount(2): Permission denied
mount.nfs4: access denied by server while mounting lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02
</pre>

You should check your systemlogs for entries like:
<pre>
Aug 07 13:09:53 systemname setroubleshoot[510523]: SELinux is preventing /usr/sbin/rpc.gssd from open access on the file /etc/krb5.keytab. For complete SELinux messages run: sealert -l XXXXXX
</pre>

or similar. To solve the issue you have to add a SELinux rule to allow access to /etc/krb5.keytab (or disable SElinux completely).

SDS@hd/Access/SMB

2024-12-06T11:51:22Z

S Siebler: /* AutoFS Setup */

SMB is a Server Message Block protocol. It has different implementations: CIFS (outdated), SMB2, SMB3, Samba

== Prerequisites ==

The SMB connection has to be established at least with protocol version SMB2.02, which is available since Windows Vista or OSX 10.7, and a NTLMv2 authentication level of "Send NTLMv2 responses only".

== Windows ==

Use a SMB share via Windows Explorer.

=== Needed Information ===
You need the following information:<br />
'''Username:''' <code>BWSERVICESAD\<username></code><br />
'''Password:''' ''ServicePassword''<br />
'''Network Path''' in UNC syntax ''':'''

<syntaxhighlight lang="bash"># for mounting the root folder
\\lsdf02.urz.uni-heidelberg.de\
# for mounting a specific SV
\\lsdf02.urz.uni-heidelberg.de\<sv-acronym> </syntaxhighlight>

=== Instructions ===

<ol style="list-style-type: decimal;">
<li><p>Open the Windows Explorer.</p></li>
<li><p>a) To establish a '''non-permanent connection''':</p>
<ul>
<li>Click on the address bar, which is located at the top of the Explorer.</li>
<li>Enter the network path and press Enter.</li></ul>

[[File:WindowsSmb0_nonPerm_cut.png|center|700px]]
<p>b) To establish a '''permanent connection''' by creating a network (pseudo) drive:</p>
<ul>
<li>Navigate to "This PC". At the top of the window, click on ''Computer'' and select ''Map network drive''.</li></ul>
[[File:windowsSmb0_connect_network_drive_cut.png|center|700px]]
</br>
<ul>
<li>Choose a drive letter to be associated with the network drive and enter the network path. Select ''use a different identification'', as these differ from your credentials used locally.</li></ul>

[[File:windowsSmb1_driveLetter.png|center|700px]]

<li><p>You will then be prompted to enter your credentials.</p>
</li>

[[File:windowsSmb2_password.png|center|x400px]]

<li><p>After logging in successfully, your network drive will appear under ''This PC''. You can now manipulate your files as accustomed.</p></li></ol>

== Mac ==

Create a network drive with Finder.

=== Needed Information ===

You need the following information:<br />
'''Username:''' <code>BWSERVICESAD\<username></code><br />
'''Password:''' ''ServicePassword''<br />
'''Network Path:''' smb://lsdf02.urz.uni-heidelberg.de/<sv-acronym>

=== Instructions ===

# Select the control field
# Select a drive letter to be associated with the network share and enter the network path. Select ‘use a different identification‘, as these differ from your credentials used locally.

Alternative: Open Finder, choose ‘Go To’ in the menu, then ‘Connect with Server’.

== Linux ==

A UNIX like operating system needs a CIFS client to use a share. CIFS clients are part of Samba implementation for Linux and other UNIX like operating systems (http://www.samba.org)

'''Attention:'''
The core CIFS protocol does not provide unix ownership information or mode for files and directories.
Because of this, files and directories will generally appear to be owned by whatever values the uid= or gid= options are set, and will have permissions set to the default file_mode and dir_mode for the mount. '''Attempting to change these values via chmod/chown will return success but have no effect.'''

For security reasons, server side permission checks cannot be overriden. The permission checks done by the server will always correspond to the credentials used to mount the share, and not necessarily to the user who is accessing the share.

Although mapping of POSIX UIDs and SIDs is not needed mounting a CIFS share '''it might become necessary when working with files on the share, e.g. when modifying ACLs'''.



=== SMB Client ===

'''Example:'''
To list the files in a SMB share, use the program smbclient.
<pre>
smbclient -U 'BWSERVICESAD\hd_xy123' //lsdf02.urz.uni-heidelberg.de/<sv-acronym>
Enter BWSERVICESAD\hd_xy123's password:
</pre>

The program allows you to access the files with a FTP like tool in an interactive shell.
<pre>
$ smbclient //lsdf02.urz.uni-heidelberg.de/<sv-acronym> -U 'BWSERVICESAD\hd_xy123'
Enter BWSERVICESAD\hd_xy123's password:
smb: \> ls
. D 0 Thu Apr 23 12:51:48 2020
.. D 0 Wed Apr 22 21:54:04 2020
bench D 0 Fri Jul 26 10:24:05 2019
benchmark_test D 0 Tue Oct 30 16:12:21 2018
checksums D 0 Mon Sep 18 10:24:21 2017
test.multiuser A 6 Thu Apr 23 12:36:07 2020
test A 7 Thu Apr 23 09:38:13 2020
.....
.snapshots DHR 0 Thu Jan 1 01:00:00 1970

115343360000 blocks of size 1024. 108260302848 blocks available
smb:\
</pre>

=== Mounting a SDS@hd Share ===

Mounting a SDS@hd CIFS share can be done by using username/password credentials or by using kerberos tickets.
Information about settting up a kerberos environment for SDS@hd can be found [[SDS@hd/Access/Kerberos|*here*]]'''.

==== Single-User Environment ====

A share can be mounted to a local directory, (e.g. /mnt/sds-hd ). Depending on your system setup, root privileges may be required.

CIFS normally binds all shares on the client as the property of the user who mounted them and transfers any existing write rights only to the user. With additional information from uid, gid, file_mode and dir_mode, other ownership and access rights can be defined when mounting on the client.

'''Nevertheless the ownership and access rights defined in this way are only simulated on the client and are not really transferred to the server.''' If access rights are changed on the client or files with other owners are created in shared folders, these changes only apply to the client and only until the next remount.

If you need to work with the correct server side permissions, please follow the setup of a [[SDS@hd/Access/CIFS#Multiuser Environment|MultiUser Setup]]

===== Mount over command line =====

'''Example:'''

<pre>
$ mkdir /mnt/sds-hd

$ sudo mount -t cifs -o username=hd_xy123,domain=BWSERVICESAD,vers=3,mfsymlinks //lsdf02.urz.uni-heidelberg.de/<sv-acronym> /mnt/sds-hd
Password:

$ df -h | grep sds-hd
//lsdf02.urz.uni-heidelberg.de/sd16j007 108T 6,6T 101T 7% /mnt/sds-hd

$ cd /mnt/sds-hd/
$ ls
</pre>
Verify the success of the mount invoking the mount command without any arguments:
<pre>
$ mount | grep lsdf02
//lsdf02.urz.uni-heidelberg.de/sd16j007 on /mnt/sds-hd type cifs (rw,relatime,vers=3.1.1,cache=strict,username=xxxx,domain=BWSERVICESAD,uid=1000,forceuid,gid=0,noforcegid,addr=xxxxx,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)
</pre>

===== Mount over /etc/fstab =====

'''Example:'''

<pre>
$ mkdir /mnt/mountpoint

/etc/fstab
//lsdf02.urz.uni-heidelberg.de/<sv_acronym> /mnt/mountpoint cifs uid=<YOUR_UID>,gid=<YOUR_GID>,user,vers=3,mfsymlinks,credentials=<path_to_user_HOME>/credentialsfile,noauto 0 0

$ cat /path_to_user_HOME/credentialsfile
username=hd_ xy123
password=<your_servicepassword>
domain=BWSERVICESAD

$ mount /mnt/mountpoint
</pre>
Verify the success of the mount invoking the mount command without any arguments:
<pre>
$ mount | grep cifs
//lsdf02.urz.uni-heidelberg.de/sd16j007 on /mnt/mountpoint type cifs (rw,relatime,vers=3.1.1,cache=strict,username=xxxx,domain=BWSERVICESAD,uid=1000,forceuid,gid=0,noforcegid,addr=xxxxx,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)
</pre>

==== Multiuser Environment ====

By default, CIFS mounts only use a single set of user credentials (the mount credentials) when accessing a share. To support different user session on the same mountpoint and the correct permission/ownership processing, the mount options <pre>multiuser,cifsacl</pre> have to be used. Because the kernel cannot prompt for passwords, '''multiuser mounts are limited to mounts using passwordless sec= options, like with sec=krb5. Information about settting up a kerberos environment can be found [[SDS@hd/Access/Kerberos|*here*]]'''

===== ID Mapping =====

In a Multiuser Environment it is important to get the correct ownerships and permissions from the server. Therefor you need to setup a [[SDS@hd/Access/ID-Mapping|ID Mapping]] environment.

Additionally we need the following packages to enable CIFS Mapping:
* RedHat/CentOS:
<pre>$ yum install cifs-utils keyutils</pre>
* debian/ubuntu:
<pre>$ apt install cifs-utils keyutils</pre>

After [[SDS@hd/Access/ID-Mapping|installing SSSD]] you have to ensure that it will be used for CIFS name resolution, e.g.

* RedHat/CentOS:
On RedHat SSSD should have allready a higher priority than winbind:
<pre>$ alternatives --display cifs-idmap-plugin

cifs-idmap-plugin - Status ist automatisch.
Link verweist auf /usr/lib64/cifs-utils/cifs_idmap_sss.so
/usr/lib64/cifs-utils/cifs_idmap_sss.so - priority 20
/usr/lib64/cifs-utils/idmapwb.so - priority 10
Zur Zeit ist die `best' Version /usr/lib64/cifs-utils/cifs_idmap_sss.so.
</pre>

* debian/ubuntu:
On debian systems SSSD has to be registered for ID mapping with an higher priority than winbind:

<pre>$ sudo update-alternatives --install /etc/cifs-utils/idmap-plugin idmap-plugin /usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so 50

$ update-alternatives --display idmap-plugin
idmap-plugin - automatischer Modus
beste Version des Links ist /usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so
Link verweist zur Zeit auf /usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so
Link idmap-plugin ist /etc/cifs-utils/idmap-plugin
Slave idmap-plugin.8.gz ist /usr/share/man/man8/idmap-plugin.8.gz
/usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so - Priorität 50
/usr/lib/x86_64-linux-gnu/cifs-utils/idmapwb.so - Priorität 40
Slave idmap-plugin.8.gz: /usr/share/man/man8/idmapwb.8.gz
</pre>

===== AutoFS Setup =====

Because CIFS shares, in contrast to nfs-Mounts, have to be mounted directly, the root user can not simply mount them into a global folder. Instead the shares have to be initially mounted by a user who has access to the Share. To achieve this, you can use the automounter "autofs".

* RedHat/CentOS:
<pre>
$ yum install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

* debian/ubuntu:
<pre>
$ apt install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

Afterwards you configure the SDS@hd Speichervorhaben in a new map file:
<pre>
$ cat /etc/auto.sds-hd
<sv-acronym1> -fstype=cifs,cifsacl,multiuser,sec=krb5,cruid=${UID},vers=3,mfsymlinks ://lsdf02.urz.uni-heidelberg.de/<sv-acronym1>
<sv-acronym2> -fstype=cifs,cifsacl,multiuser,sec=krb5,cruid=${UID},vers=3,mfsymlinks ://lsdf02.urz.uni-heidelberg.de/<sv-acronym2>
....
</pre>

You have to include the new map into the auto.master file:
<pre>
$ cat /etc/auto.master
[...]
/mnt/sds-hd /etc/auto.sds-hd
[...]
</pre>

To display all available SDS@hd shares on this machine to the users, you should enable "browser_mode":
<pre>
$ cat /etc/autofs.conf
[...]
# to display all available SDS-hd shares on this to the users
browse_mode=yes
[...]
</pre>
otherwise each share-folder will only be visible after a user has mounted.

After changing the configuration, you should restart the autofs daemon, e.g.:
<pre>
$ systemctl restart autofs
</pre>

Of course you can adopt all other autofs options, like timeouts, etc. to the specific needs of your environment or use any other method for dynamically mounting the CIFS shares.

===== Access the Share =====

Now each user should be able to mount a SDS@hd share, which is configured for the machine. If a share is allready mounted, other users will access this share with their own credentials without mounting again.

To get access, each user needs a valid kerberos ticket, which can be fetched with
<pre>
$ kinit hd_xy123
</pre>

For further information about handling kerberos tickets take a look at [[SDS@hd/Access/NFS#Access_your_data|SDS@hd kerberos]]

SDS@hd/Access/SMB

2024-12-06T11:50:37Z

S Siebler: /* Mount over /etc/fstab */

SMB is a Server Message Block protocol. It has different implementations: CIFS (outdated), SMB2, SMB3, Samba

== Prerequisites ==

The SMB connection has to be established at least with protocol version SMB2.02, which is available since Windows Vista or OSX 10.7, and a NTLMv2 authentication level of "Send NTLMv2 responses only".

== Windows ==

Use a SMB share via Windows Explorer.

=== Needed Information ===
You need the following information:<br />
'''Username:''' <code>BWSERVICESAD\<username></code><br />
'''Password:''' ''ServicePassword''<br />
'''Network Path''' in UNC syntax ''':'''

<syntaxhighlight lang="bash"># for mounting the root folder
\\lsdf02.urz.uni-heidelberg.de\
# for mounting a specific SV
\\lsdf02.urz.uni-heidelberg.de\<sv-acronym> </syntaxhighlight>

=== Instructions ===

<ol style="list-style-type: decimal;">
<li><p>Open the Windows Explorer.</p></li>
<li><p>a) To establish a '''non-permanent connection''':</p>
<ul>
<li>Click on the address bar, which is located at the top of the Explorer.</li>
<li>Enter the network path and press Enter.</li></ul>

[[File:WindowsSmb0_nonPerm_cut.png|center|700px]]
<p>b) To establish a '''permanent connection''' by creating a network (pseudo) drive:</p>
<ul>
<li>Navigate to "This PC". At the top of the window, click on ''Computer'' and select ''Map network drive''.</li></ul>
[[File:windowsSmb0_connect_network_drive_cut.png|center|700px]]
</br>
<ul>
<li>Choose a drive letter to be associated with the network drive and enter the network path. Select ''use a different identification'', as these differ from your credentials used locally.</li></ul>

[[File:windowsSmb1_driveLetter.png|center|700px]]

<li><p>You will then be prompted to enter your credentials.</p>
</li>

[[File:windowsSmb2_password.png|center|x400px]]

<li><p>After logging in successfully, your network drive will appear under ''This PC''. You can now manipulate your files as accustomed.</p></li></ol>

== Mac ==

Create a network drive with Finder.

=== Needed Information ===

You need the following information:<br />
'''Username:''' <code>BWSERVICESAD\<username></code><br />
'''Password:''' ''ServicePassword''<br />
'''Network Path:''' smb://lsdf02.urz.uni-heidelberg.de/<sv-acronym>

=== Instructions ===

# Select the control field
# Select a drive letter to be associated with the network share and enter the network path. Select ‘use a different identification‘, as these differ from your credentials used locally.

Alternative: Open Finder, choose ‘Go To’ in the menu, then ‘Connect with Server’.

== Linux ==

A UNIX like operating system needs a CIFS client to use a share. CIFS clients are part of Samba implementation for Linux and other UNIX like operating systems (http://www.samba.org)

'''Attention:'''
The core CIFS protocol does not provide unix ownership information or mode for files and directories.
Because of this, files and directories will generally appear to be owned by whatever values the uid= or gid= options are set, and will have permissions set to the default file_mode and dir_mode for the mount. '''Attempting to change these values via chmod/chown will return success but have no effect.'''

For security reasons, server side permission checks cannot be overriden. The permission checks done by the server will always correspond to the credentials used to mount the share, and not necessarily to the user who is accessing the share.

Although mapping of POSIX UIDs and SIDs is not needed mounting a CIFS share '''it might become necessary when working with files on the share, e.g. when modifying ACLs'''.



=== SMB Client ===

'''Example:'''
To list the files in a SMB share, use the program smbclient.
<pre>
smbclient -U 'BWSERVICESAD\hd_xy123' //lsdf02.urz.uni-heidelberg.de/<sv-acronym>
Enter BWSERVICESAD\hd_xy123's password:
</pre>

The program allows you to access the files with a FTP like tool in an interactive shell.
<pre>
$ smbclient //lsdf02.urz.uni-heidelberg.de/<sv-acronym> -U 'BWSERVICESAD\hd_xy123'
Enter BWSERVICESAD\hd_xy123's password:
smb: \> ls
. D 0 Thu Apr 23 12:51:48 2020
.. D 0 Wed Apr 22 21:54:04 2020
bench D 0 Fri Jul 26 10:24:05 2019
benchmark_test D 0 Tue Oct 30 16:12:21 2018
checksums D 0 Mon Sep 18 10:24:21 2017
test.multiuser A 6 Thu Apr 23 12:36:07 2020
test A 7 Thu Apr 23 09:38:13 2020
.....
.snapshots DHR 0 Thu Jan 1 01:00:00 1970

115343360000 blocks of size 1024. 108260302848 blocks available
smb:\
</pre>

=== Mounting a SDS@hd Share ===

Mounting a SDS@hd CIFS share can be done by using username/password credentials or by using kerberos tickets.
Information about settting up a kerberos environment for SDS@hd can be found [[SDS@hd/Access/Kerberos|*here*]]'''.

==== Single-User Environment ====

A share can be mounted to a local directory, (e.g. /mnt/sds-hd ). Depending on your system setup, root privileges may be required.

CIFS normally binds all shares on the client as the property of the user who mounted them and transfers any existing write rights only to the user. With additional information from uid, gid, file_mode and dir_mode, other ownership and access rights can be defined when mounting on the client.

'''Nevertheless the ownership and access rights defined in this way are only simulated on the client and are not really transferred to the server.''' If access rights are changed on the client or files with other owners are created in shared folders, these changes only apply to the client and only until the next remount.

If you need to work with the correct server side permissions, please follow the setup of a [[SDS@hd/Access/CIFS#Multiuser Environment|MultiUser Setup]]

===== Mount over command line =====

'''Example:'''

<pre>
$ mkdir /mnt/sds-hd

$ sudo mount -t cifs -o username=hd_xy123,domain=BWSERVICESAD,vers=3,mfsymlinks //lsdf02.urz.uni-heidelberg.de/<sv-acronym> /mnt/sds-hd
Password:

$ df -h | grep sds-hd
//lsdf02.urz.uni-heidelberg.de/sd16j007 108T 6,6T 101T 7% /mnt/sds-hd

$ cd /mnt/sds-hd/
$ ls
</pre>
Verify the success of the mount invoking the mount command without any arguments:
<pre>
$ mount | grep lsdf02
//lsdf02.urz.uni-heidelberg.de/sd16j007 on /mnt/sds-hd type cifs (rw,relatime,vers=3.1.1,cache=strict,username=xxxx,domain=BWSERVICESAD,uid=1000,forceuid,gid=0,noforcegid,addr=xxxxx,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)
</pre>

===== Mount over /etc/fstab =====

'''Example:'''

<pre>
$ mkdir /mnt/mountpoint

/etc/fstab
//lsdf02.urz.uni-heidelberg.de/<sv_acronym> /mnt/mountpoint cifs uid=<YOUR_UID>,gid=<YOUR_GID>,user,vers=3,mfsymlinks,credentials=<path_to_user_HOME>/credentialsfile,noauto 0 0

$ cat /path_to_user_HOME/credentialsfile
username=hd_ xy123
password=<your_servicepassword>
domain=BWSERVICESAD

$ mount /mnt/mountpoint
</pre>
Verify the success of the mount invoking the mount command without any arguments:
<pre>
$ mount | grep cifs
//lsdf02.urz.uni-heidelberg.de/sd16j007 on /mnt/mountpoint type cifs (rw,relatime,vers=3.1.1,cache=strict,username=xxxx,domain=BWSERVICESAD,uid=1000,forceuid,gid=0,noforcegid,addr=xxxxx,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)
</pre>

==== Multiuser Environment ====

By default, CIFS mounts only use a single set of user credentials (the mount credentials) when accessing a share. To support different user session on the same mountpoint and the correct permission/ownership processing, the mount options <pre>multiuser,cifsacl</pre> have to be used. Because the kernel cannot prompt for passwords, '''multiuser mounts are limited to mounts using passwordless sec= options, like with sec=krb5. Information about settting up a kerberos environment can be found [[SDS@hd/Access/Kerberos|*here*]]'''

===== ID Mapping =====

In a Multiuser Environment it is important to get the correct ownerships and permissions from the server. Therefor you need to setup a [[SDS@hd/Access/ID-Mapping|ID Mapping]] environment.

Additionally we need the following packages to enable CIFS Mapping:
* RedHat/CentOS:
<pre>$ yum install cifs-utils keyutils</pre>
* debian/ubuntu:
<pre>$ apt install cifs-utils keyutils</pre>

After [[SDS@hd/Access/ID-Mapping|installing SSSD]] you have to ensure that it will be used for CIFS name resolution, e.g.

* RedHat/CentOS:
On RedHat SSSD should have allready a higher priority than winbind:
<pre>$ alternatives --display cifs-idmap-plugin

cifs-idmap-plugin - Status ist automatisch.
Link verweist auf /usr/lib64/cifs-utils/cifs_idmap_sss.so
/usr/lib64/cifs-utils/cifs_idmap_sss.so - priority 20
/usr/lib64/cifs-utils/idmapwb.so - priority 10
Zur Zeit ist die `best' Version /usr/lib64/cifs-utils/cifs_idmap_sss.so.
</pre>

* debian/ubuntu:
On debian systems SSSD has to be registered for ID mapping with an higher priority than winbind:

<pre>$ sudo update-alternatives --install /etc/cifs-utils/idmap-plugin idmap-plugin /usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so 50

$ update-alternatives --display idmap-plugin
idmap-plugin - automatischer Modus
beste Version des Links ist /usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so
Link verweist zur Zeit auf /usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so
Link idmap-plugin ist /etc/cifs-utils/idmap-plugin
Slave idmap-plugin.8.gz ist /usr/share/man/man8/idmap-plugin.8.gz
/usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so - Priorität 50
/usr/lib/x86_64-linux-gnu/cifs-utils/idmapwb.so - Priorität 40
Slave idmap-plugin.8.gz: /usr/share/man/man8/idmapwb.8.gz
</pre>

===== AutoFS Setup =====

Because CIFS shares, in contrast to nfs-Mounts, have to be mounted directly, the root user can not simply mount them into a global folder. Instead the shares have to be initially mounted by a user who has access to the Share. To achieve this, you can use the automounter "autofs".

* RedHat/CentOS:
<pre>
$ yum install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

* debian/ubuntu:
<pre>
$ apt install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

Afterwards you configure the SDS@hd Speichervorhaben in a new map file:
<pre>
$ cat /etc/auto.sds-hd
<sv-acronym1> -fstype=cifs,cifsacl,multiuser,sec=krb5,cruid=${UID},vers=3.0,mfsymlinks ://lsdf02.urz.uni-heidelberg.de/<sv-acronym1>
<sv-acronym2> -fstype=cifs,cifsacl,multiuser,sec=krb5,cruid=${UID},vers=3.0,mfsymlinks ://lsdf02.urz.uni-heidelberg.de/<sv-acronym2>
....
</pre>

You have to include the new map into the auto.master file:
<pre>
$ cat /etc/auto.master
[...]
/mnt/sds-hd /etc/auto.sds-hd
[...]
</pre>

To display all available SDS@hd shares on this machine to the users, you should enable "browser_mode":
<pre>
$ cat /etc/autofs.conf
[...]
# to display all available SDS-hd shares on this to the users
browse_mode=yes
[...]
</pre>
otherwise each share-folder will only be visible after a user has mounted.

After changing the configuration, you should restart the autofs daemon, e.g.:
<pre>
$ systemctl restart autofs
</pre>

Of course you can adopt all other autofs options, like timeouts, etc. to the specific needs of your environment or use any other method for dynamically mounting the CIFS shares.

===== Access the Share =====

Now each user should be able to mount a SDS@hd share, which is configured for the machine. If a share is allready mounted, other users will access this share with their own credentials without mounting again.

To get access, each user needs a valid kerberos ticket, which can be fetched with
<pre>
$ kinit hd_xy123
</pre>

For further information about handling kerberos tickets take a look at [[SDS@hd/Access/NFS#Access_your_data|SDS@hd kerberos]]

SDS@hd/Access/SMB

2024-12-06T11:50:18Z

S Siebler: /* Mount over command line */

SMB is a Server Message Block protocol. It has different implementations: CIFS (outdated), SMB2, SMB3, Samba

== Prerequisites ==

The SMB connection has to be established at least with protocol version SMB2.02, which is available since Windows Vista or OSX 10.7, and a NTLMv2 authentication level of "Send NTLMv2 responses only".

== Windows ==

Use a SMB share via Windows Explorer.

=== Needed Information ===
You need the following information:<br />
'''Username:''' <code>BWSERVICESAD\<username></code><br />
'''Password:''' ''ServicePassword''<br />
'''Network Path''' in UNC syntax ''':'''

<syntaxhighlight lang="bash"># for mounting the root folder
\\lsdf02.urz.uni-heidelberg.de\
# for mounting a specific SV
\\lsdf02.urz.uni-heidelberg.de\<sv-acronym> </syntaxhighlight>

=== Instructions ===

<ol style="list-style-type: decimal;">
<li><p>Open the Windows Explorer.</p></li>
<li><p>a) To establish a '''non-permanent connection''':</p>
<ul>
<li>Click on the address bar, which is located at the top of the Explorer.</li>
<li>Enter the network path and press Enter.</li></ul>

[[File:WindowsSmb0_nonPerm_cut.png|center|700px]]
<p>b) To establish a '''permanent connection''' by creating a network (pseudo) drive:</p>
<ul>
<li>Navigate to "This PC". At the top of the window, click on ''Computer'' and select ''Map network drive''.</li></ul>
[[File:windowsSmb0_connect_network_drive_cut.png|center|700px]]
</br>
<ul>
<li>Choose a drive letter to be associated with the network drive and enter the network path. Select ''use a different identification'', as these differ from your credentials used locally.</li></ul>

[[File:windowsSmb1_driveLetter.png|center|700px]]

<li><p>You will then be prompted to enter your credentials.</p>
</li>

[[File:windowsSmb2_password.png|center|x400px]]

<li><p>After logging in successfully, your network drive will appear under ''This PC''. You can now manipulate your files as accustomed.</p></li></ol>

== Mac ==

Create a network drive with Finder.

=== Needed Information ===

You need the following information:<br />
'''Username:''' <code>BWSERVICESAD\<username></code><br />
'''Password:''' ''ServicePassword''<br />
'''Network Path:''' smb://lsdf02.urz.uni-heidelberg.de/<sv-acronym>

=== Instructions ===

# Select the control field
# Select a drive letter to be associated with the network share and enter the network path. Select ‘use a different identification‘, as these differ from your credentials used locally.

Alternative: Open Finder, choose ‘Go To’ in the menu, then ‘Connect with Server’.

== Linux ==

A UNIX like operating system needs a CIFS client to use a share. CIFS clients are part of Samba implementation for Linux and other UNIX like operating systems (http://www.samba.org)

'''Attention:'''
The core CIFS protocol does not provide unix ownership information or mode for files and directories.
Because of this, files and directories will generally appear to be owned by whatever values the uid= or gid= options are set, and will have permissions set to the default file_mode and dir_mode for the mount. '''Attempting to change these values via chmod/chown will return success but have no effect.'''

For security reasons, server side permission checks cannot be overriden. The permission checks done by the server will always correspond to the credentials used to mount the share, and not necessarily to the user who is accessing the share.

Although mapping of POSIX UIDs and SIDs is not needed mounting a CIFS share '''it might become necessary when working with files on the share, e.g. when modifying ACLs'''.



=== SMB Client ===

'''Example:'''
To list the files in a SMB share, use the program smbclient.
<pre>
smbclient -U 'BWSERVICESAD\hd_xy123' //lsdf02.urz.uni-heidelberg.de/<sv-acronym>
Enter BWSERVICESAD\hd_xy123's password:
</pre>

The program allows you to access the files with a FTP like tool in an interactive shell.
<pre>
$ smbclient //lsdf02.urz.uni-heidelberg.de/<sv-acronym> -U 'BWSERVICESAD\hd_xy123'
Enter BWSERVICESAD\hd_xy123's password:
smb: \> ls
. D 0 Thu Apr 23 12:51:48 2020
.. D 0 Wed Apr 22 21:54:04 2020
bench D 0 Fri Jul 26 10:24:05 2019
benchmark_test D 0 Tue Oct 30 16:12:21 2018
checksums D 0 Mon Sep 18 10:24:21 2017
test.multiuser A 6 Thu Apr 23 12:36:07 2020
test A 7 Thu Apr 23 09:38:13 2020
.....
.snapshots DHR 0 Thu Jan 1 01:00:00 1970

115343360000 blocks of size 1024. 108260302848 blocks available
smb:\
</pre>

=== Mounting a SDS@hd Share ===

Mounting a SDS@hd CIFS share can be done by using username/password credentials or by using kerberos tickets.
Information about settting up a kerberos environment for SDS@hd can be found [[SDS@hd/Access/Kerberos|*here*]]'''.

==== Single-User Environment ====

A share can be mounted to a local directory, (e.g. /mnt/sds-hd ). Depending on your system setup, root privileges may be required.

CIFS normally binds all shares on the client as the property of the user who mounted them and transfers any existing write rights only to the user. With additional information from uid, gid, file_mode and dir_mode, other ownership and access rights can be defined when mounting on the client.

'''Nevertheless the ownership and access rights defined in this way are only simulated on the client and are not really transferred to the server.''' If access rights are changed on the client or files with other owners are created in shared folders, these changes only apply to the client and only until the next remount.

If you need to work with the correct server side permissions, please follow the setup of a [[SDS@hd/Access/CIFS#Multiuser Environment|MultiUser Setup]]

===== Mount over command line =====

'''Example:'''

<pre>
$ mkdir /mnt/sds-hd

$ sudo mount -t cifs -o username=hd_xy123,domain=BWSERVICESAD,vers=3,mfsymlinks //lsdf02.urz.uni-heidelberg.de/<sv-acronym> /mnt/sds-hd
Password:

$ df -h | grep sds-hd
//lsdf02.urz.uni-heidelberg.de/sd16j007 108T 6,6T 101T 7% /mnt/sds-hd

$ cd /mnt/sds-hd/
$ ls
</pre>
Verify the success of the mount invoking the mount command without any arguments:
<pre>
$ mount | grep lsdf02
//lsdf02.urz.uni-heidelberg.de/sd16j007 on /mnt/sds-hd type cifs (rw,relatime,vers=3.1.1,cache=strict,username=xxxx,domain=BWSERVICESAD,uid=1000,forceuid,gid=0,noforcegid,addr=xxxxx,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)
</pre>

===== Mount over /etc/fstab =====

'''Example:'''

<pre>
$ mkdir /mnt/mountpoint

/etc/fstab
//lsdf02.urz.uni-heidelberg.de/<sv_acronym> /mnt/mountpoint cifs uid=<YOUR_UID>,gid=<YOUR_GID>,user,vers=3.0,mfsymlinks,credentials=<path_to_user_HOME>/credentialsfile,noauto 0 0

$ cat /path_to_user_HOME/credentialsfile
username=hd_ xy123
password=<your_servicepassword>
domain=BWSERVICESAD

$ mount /mnt/mountpoint
</pre>
Verify the success of the mount invoking the mount command without any arguments:
<pre>
$ mount | grep cifs
//lsdf02.urz.uni-heidelberg.de/sd16j007 on /mnt/mountpoint type cifs (rw,relatime,vers=3.0,cache=strict,username=xxxx,domain=BWSERVICESAD,uid=1000,forceuid,gid=0,noforcegid,addr=xxxxx,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)
</pre>

==== Multiuser Environment ====

By default, CIFS mounts only use a single set of user credentials (the mount credentials) when accessing a share. To support different user session on the same mountpoint and the correct permission/ownership processing, the mount options <pre>multiuser,cifsacl</pre> have to be used. Because the kernel cannot prompt for passwords, '''multiuser mounts are limited to mounts using passwordless sec= options, like with sec=krb5. Information about settting up a kerberos environment can be found [[SDS@hd/Access/Kerberos|*here*]]'''

===== ID Mapping =====

In a Multiuser Environment it is important to get the correct ownerships and permissions from the server. Therefor you need to setup a [[SDS@hd/Access/ID-Mapping|ID Mapping]] environment.

Additionally we need the following packages to enable CIFS Mapping:
* RedHat/CentOS:
<pre>$ yum install cifs-utils keyutils</pre>
* debian/ubuntu:
<pre>$ apt install cifs-utils keyutils</pre>

After [[SDS@hd/Access/ID-Mapping|installing SSSD]] you have to ensure that it will be used for CIFS name resolution, e.g.

* RedHat/CentOS:
On RedHat SSSD should have allready a higher priority than winbind:
<pre>$ alternatives --display cifs-idmap-plugin

cifs-idmap-plugin - Status ist automatisch.
Link verweist auf /usr/lib64/cifs-utils/cifs_idmap_sss.so
/usr/lib64/cifs-utils/cifs_idmap_sss.so - priority 20
/usr/lib64/cifs-utils/idmapwb.so - priority 10
Zur Zeit ist die `best' Version /usr/lib64/cifs-utils/cifs_idmap_sss.so.
</pre>

* debian/ubuntu:
On debian systems SSSD has to be registered for ID mapping with an higher priority than winbind:

<pre>$ sudo update-alternatives --install /etc/cifs-utils/idmap-plugin idmap-plugin /usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so 50

$ update-alternatives --display idmap-plugin
idmap-plugin - automatischer Modus
beste Version des Links ist /usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so
Link verweist zur Zeit auf /usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so
Link idmap-plugin ist /etc/cifs-utils/idmap-plugin
Slave idmap-plugin.8.gz ist /usr/share/man/man8/idmap-plugin.8.gz
/usr/lib/x86_64-linux-gnu/cifs-utils/cifs_idmap_sss.so - Priorität 50
/usr/lib/x86_64-linux-gnu/cifs-utils/idmapwb.so - Priorität 40
Slave idmap-plugin.8.gz: /usr/share/man/man8/idmapwb.8.gz
</pre>

===== AutoFS Setup =====

Because CIFS shares, in contrast to nfs-Mounts, have to be mounted directly, the root user can not simply mount them into a global folder. Instead the shares have to be initially mounted by a user who has access to the Share. To achieve this, you can use the automounter "autofs".

* RedHat/CentOS:
<pre>
$ yum install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

* debian/ubuntu:
<pre>
$ apt install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

Afterwards you configure the SDS@hd Speichervorhaben in a new map file:
<pre>
$ cat /etc/auto.sds-hd
<sv-acronym1> -fstype=cifs,cifsacl,multiuser,sec=krb5,cruid=${UID},vers=3.0,mfsymlinks ://lsdf02.urz.uni-heidelberg.de/<sv-acronym1>
<sv-acronym2> -fstype=cifs,cifsacl,multiuser,sec=krb5,cruid=${UID},vers=3.0,mfsymlinks ://lsdf02.urz.uni-heidelberg.de/<sv-acronym2>
....
</pre>

You have to include the new map into the auto.master file:
<pre>
$ cat /etc/auto.master
[...]
/mnt/sds-hd /etc/auto.sds-hd
[...]
</pre>

To display all available SDS@hd shares on this machine to the users, you should enable "browser_mode":
<pre>
$ cat /etc/autofs.conf
[...]
# to display all available SDS-hd shares on this to the users
browse_mode=yes
[...]
</pre>
otherwise each share-folder will only be visible after a user has mounted.

After changing the configuration, you should restart the autofs daemon, e.g.:
<pre>
$ systemctl restart autofs
</pre>

Of course you can adopt all other autofs options, like timeouts, etc. to the specific needs of your environment or use any other method for dynamically mounting the CIFS shares.

===== Access the Share =====

Now each user should be able to mount a SDS@hd share, which is configured for the machine. If a share is allready mounted, other users will access this share with their own credentials without mounting again.

To get access, each user needs a valid kerberos ticket, which can be fetched with
<pre>
$ kinit hd_xy123
</pre>

For further information about handling kerberos tickets take a look at [[SDS@hd/Access/NFS#Access_your_data|SDS@hd kerberos]]

SDS@hd/Access/NFS

2024-08-07T11:31:15Z

S Siebler:

= <b> Prerequisites </b> =

* '''Attention:''' To access data served by SDS@hd, you need a '''''Service Password'''''. See details [[SDS@hd/Registration]].

* Additionally the access to SDS@hd is currently only available inside the [https://www.belwue.de/netz/netz0.html belwue-Network]. This means you have to use the VPN Service of your HomeOrganization, if you want to access SDS@hd from outside the bwHPC-Clusters (e.g. via [https://www.eduroam.org/where/ eduroam] or from your personal Laptop)

* The access via nfs protocol is machine-based, which means '''new nfs-Clients have to be registered''' on SDS@hd. During this registration each machine receives a keytab file from the SDS@hd team, which allows mounting SDS@hd. In order the keytab file to be created, please [mailto:sds-hd-support@urz.uni-heidelberg.de?subject=SDS@hd%20nfs-Client%20Registration send an email] to SDS@hd team for Clientregistration with the following information:
** hostname of the new nfs-Client
** IP address
** short description
** location
** acronym of the Speichervorhaben which should be available on this machine

= <b> Using NFSv4 for UNIX client </b> =

The authentication for data access via NFSv4 is performed using Kerberostickets. This requires a functioning Kerberos environment on the client!

{{:SDS@hd/Access/Kerberos}}

After configuring kerberos, you have to install nfs packages in your system, and enable kerberized NFSv4. The exact names of the packages depending on you linux distribution (see examples below).

''Example RedHat/CentOS''
<pre>
> yum install nfs-utils nfs4-acl-tools

/etc/sysconfig/nfs:
NEED_IDMAPD=yes
NEED_GSSD=yes
</pre>

''Example debian/ubuntu''
<pre>
> apt install nfs-common nfs4-acl-tools nfs-server

/etc/default/nfs-common:
NEED_IDMAPD=yes
NEED_GSSD=yes
</pre>
On ubuntu server: nfs-kernel-server

{{:SDS@hd/Access/ID-Mapping}}

To enable the ID-Mapping for NFSv4 mounts change the file ''/etc/idmapd.conf'' with the following lines:
<pre>
in /etc/idmapd.conf:
[General]
Domain = urz.uni-heidelberg.de
Local-Realms = BWSERVICES.UNI-HEIDELBERG.DE
</pre>

== mount a nfs share ==
The usual restrictions for mounting drives under Linux apply. Usually this can only be done by the superuser "root". For detailed information, please contact the system administrator of your system.

After successfull configuration (s. 2.1) you can mount your SDS@hd share with the following commands:
<pre>
> mkdir <mountpoint>
> mount -t nfs4 -o sec=krb5,vers=4.1 lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02/ <mountpoint>
</pre>

To enable the mounting after a restart, you have to add the following line to the file "/etc/fstab"
<pre>
lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02/ <mountpoint> nfs4 sec=krb5,vers=4.1 0 0
</pre>

=== AutoFS Setup ===

Instead of the fstab-entry you can also use the automounter "autofs".

* RedHat/CentOS:
<pre>
$ yum install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

* debian/ubuntu:
<pre>
$ apt install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

Afterwards you configure the SDS@hd Speichervorhaben in a new map file:
<pre>
$ cat /etc/auto.sds-hd
sds-hd -fstype=nfs4,rw,sec=krb5,vers=4.1,nosuid,nodev lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02
....
</pre>

You have to include the new map into the auto.master file, e.g.:
<pre>
$ cat /etc/auto.master
[...]
/mnt /etc/auto.sds-hd
[...]
</pre>

To display all available SDS@hd shares on this machine to the users, you should enable "browser_mode":
<pre>
$ cat /etc/autofs.conf
[...]
# to display all available SDS-hd shares on this to the users
browse_mode=yes
[...]
</pre>
otherwise each share-folder will only be visible after a user has mounted.

After changing the configuration, you should restart the autofs daemon, e.g.:
<pre>
$ systemctl restart autofs
</pre>

Of course you can adopt all other autofs options, like timeouts, etc. to the specific needs of your environment or use any other method for dynamically mounting the shares.

== access your data ==
'''Attention!''' The access can not be done as root user, because root uses the Kerberosticket of the machine, which does not have data access!

To access your data on SDS@hd you have to fetch a valid kerberos ticket with your SDS@hd user and Servicepassword:
<pre>
> kinit hd_xy123
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
</pre>
You can check afterwards your kerberos ticket with:
<pre>
> klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE

Valid starting Expires Service principal
20.09.2017 04:00:01 21.09.2017 04:00:01 krbtgt/BWSERVICES.UNI-HEIDELBERG.DE@BWSERVICES.UNI-HEIDELBERG.DE
renew until 29.09.2017 13:38:49
</pre>

Afterwards you should be able to access the mountpoint, which contain all Speichervorhaben exported to your machine:
<pre>
> ls <mountpoint>
sd16j007 sd17c010 sd17d005
</pre>

== renew a kerberos ticket ==
Because a kerberos ticket has a limited lifetime (default: 10 hours, maximum 24 hours) for security reasons, you have to renew your ticket before it expires to prevent access loss.
<pre>
> kinit -R
</pre>

This renewal could only be done for maximum time of 10 Days and as long as the current kerberos ticket is still valid. For renewal of an expired ticket, you have to use again your Servicepassword.

== destroy kerberos ticket ==
Even if kerberos tickets are only valid for a limited period of time, a ticket should be destroyed as soon as access is no longer needed to prevent misuse on multi-user systems:
<pre>kdestroy</pre>

== automated kerberos tickets ==
<span style="color:red"><strong>'''Attention!''' Keep this generated Keytab safe and use it only in trusted environments!</strong></span>

If your workflow needs a permanent access to SDS@hd for longer than 10 Days, you can use '''ktutil''' to encrypt your Service Password into a keytab file:

''interactive way:''
<pre>
ktutil
ktutil: addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e rc4-hmac
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
ktutil: addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e aes256-cts
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
ktutil: wkt xy123.keytab
ktuitl: quit
</pre>

''non-interactive way:''
<pre>
echo -e "addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e rc4-hmac\n<your_servicepasword>\n
addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e aes256-cts\n<your_servicepasword>\nwkt xy123.keytab" | ktutil
</pre>

With this keytab, you can fetch a kerberos ticket without an interactive password:
<pre>
kinit -k -t xy123.keytab hd_xy123
</pre>
<br>

= <b> Troubleshooting </b> =

== incorrect mount option ==

When you try to mount SDS@hd you get the following error message:
<pre>
mount.nfs4: mount(2): Invalid argument
mount.nfs4: an incorrect mount option was specified
</pre>

This means the service rpc-gssd.service is not yet running. Please try to start the service via <pre>systemctl restart rpc-gssd.service</pre>

== permission denied while mounting when SELinux is enabled ==

When trying to mount, you get the following error message:
<pre>
mount.nfs4: mount(2): Permission denied
mount.nfs4: trying text-based options 'sec=krb5,vers=4.1,addr=x.x.x.x,clientaddr=a.b.c.d'
mount.nfs4: mount(2): Permission denied
mount.nfs4: access denied by server while mounting lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02
</pre>

You should check your systemlogs for entries like:
<pre>
Aug 07 13:09:53 systemname setroubleshoot[510523]: SELinux is preventing /usr/sbin/rpc.gssd from open access on the file /etc/krb5.keytab. For complete SELinux messages run: sealert -l XXXXXX
</pre>

or similar. To solve the issue you have to add a SELinux rule to allow access to /etc/krb5.keytab (or disable SElinux completely).

SDS@hd/Access/NFS

2024-08-07T11:28:18Z

S Siebler:

= <b> Prerequisites </b> =

* '''Attention:''' To access data served by SDS@hd, you need a '''''Service Password'''''. See details [[SDS@hd/Registration]].

* Additionally the access to SDS@hd is currently only available inside the [https://www.belwue.de/netz/netz0.html belwue-Network]. This means you have to use the VPN Service of your HomeOrganization, if you want to access SDS@hd from outside the bwHPC-Clusters (e.g. via [https://www.eduroam.org/where/ eduroam] or from your personal Laptop)

* The access via nfs protocol is machine-based, which means '''new nfs-Clients have to be registered''' on SDS@hd. During this registration each machine receives a keytab file from the SDS@hd team, which allows mounting SDS@hd. In order the keytab file to be created, please [mailto:sds-hd-support@urz.uni-heidelberg.de?subject=SDS@hd%20nfs-Client%20Registration send an email] to SDS@hd team for Clientregistration with the following information:
** hostname of the new nfs-Client
** IP address
** short description
** location
** acronym of the Speichervorhaben which should be available on this machine

= <b> Using NFSv4 for UNIX client </b> =

The authentication for data access via NFSv4 is performed using Kerberostickets. This requires a functioning Kerberos environment on the client!

{{:SDS@hd/Access/Kerberos}}

After configuring kerberos, you have to install nfs packages in your system, and enable kerberized NFSv4. The exact names of the packages depending on you linux distribution (see examples below).

''Example RedHat/CentOS''
<pre>
> yum install nfs-utils nfs4-acl-tools

/etc/sysconfig/nfs:
NEED_IDMAPD=yes
NEED_GSSD=yes
</pre>

''Example debian/ubuntu''
<pre>
> apt install nfs-common nfs4-acl-tools nfs-server

/etc/default/nfs-common:
NEED_IDMAPD=yes
NEED_GSSD=yes
</pre>
On ubuntu server: nfs-kernel-server

{{:SDS@hd/Access/ID-Mapping}}

To enable the ID-Mapping for NFSv4 mounts change the file ''/etc/idmapd.conf'' with the following lines:
<pre>
in /etc/idmapd.conf:
[General]
Domain = urz.uni-heidelberg.de
Local-Realms = BWSERVICES.UNI-HEIDELBERG.DE
</pre>

== mount a nfs share ==
The usual restrictions for mounting drives under Linux apply. Usually this can only be done by the superuser "root". For detailed information, please contact the system administrator of your system.

After successfull configuration (s. 2.1) you can mount your SDS@hd share with the following commands:
<pre>
> mkdir <mountpoint>
> mount -t nfs4 -o sec=krb5,vers=4.1 lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02/ <mountpoint>
</pre>

To enable the mounting after a restart, you have to add the following line to the file "/etc/fstab"
<pre>
lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02/ <mountpoint> nfs4 sec=krb5,vers=4.1 0 0
</pre>

=== AutoFS Setup ===

Instead of the fstab-entry you can also use the automounter "autofs".

* RedHat/CentOS:
<pre>
$ yum install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

* debian/ubuntu:
<pre>
$ apt install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

Afterwards you configure the SDS@hd Speichervorhaben in a new map file:
<pre>
$ cat /etc/auto.sds-hd
sds-hd -fstype=nfs4,rw,sec=krb5,vers=4.1,nosuid,nodev lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02
....
</pre>

You have to include the new map into the auto.master file, e.g.:
<pre>
$ cat /etc/auto.master
[...]
/mnt /etc/auto.sds-hd
[...]
</pre>

To display all available SDS@hd shares on this machine to the users, you should enable "browser_mode":
<pre>
$ cat /etc/autofs.conf
[...]
# to display all available SDS-hd shares on this to the users
browse_mode=yes
[...]
</pre>
otherwise each share-folder will only be visible after a user has mounted.

After changing the configuration, you should restart the autofs daemon, e.g.:
<pre>
$ systemctl restart autofs
</pre>

Of course you can adopt all other autofs options, like timeouts, etc. to the specific needs of your environment or use any other method for dynamically mounting the shares.

== access your data ==
'''Attention!''' The access can not be done as root user, because root uses the Kerberosticket of the machine, which does not have data access!

To access your data on SDS@hd you have to fetch a valid kerberos ticket with your SDS@hd user and Servicepassword:
<pre>
> kinit hd_xy123
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
</pre>
You can check afterwards your kerberos ticket with:
<pre>
> klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE

Valid starting Expires Service principal
20.09.2017 04:00:01 21.09.2017 04:00:01 krbtgt/BWSERVICES.UNI-HEIDELBERG.DE@BWSERVICES.UNI-HEIDELBERG.DE
renew until 29.09.2017 13:38:49
</pre>

Afterwards you should be able to access the mountpoint, which contain all Speichervorhaben exported to your machine:
<pre>
> ls <mountpoint>
sd16j007 sd17c010 sd17d005
</pre>

== renew a kerberos ticket ==
Because a kerberos ticket has a limited lifetime (default: 10 hours, maximum 24 hours) for security reasons, you have to renew your ticket before it expires to prevent access loss.
<pre>
> kinit -R
</pre>

This renewal could only be done for maximum time of 10 Days and as long as the current kerberos ticket is still valid. For renewal of an expired ticket, you have to use again your Servicepassword.

== destroy kerberos ticket ==
Even if kerberos tickets are only valid for a limited period of time, a ticket should be destroyed as soon as access is no longer needed to prevent misuse on multi-user systems:
<pre>kdestroy</pre>

== automated kerberos tickets ==
<span style="color:red"><strong>'''Attention!''' Keep this generated Keytab safe and use it only in trusted environments!</strong></span>

If your workflow needs a permanent access to SDS@hd for longer than 10 Days, you can use '''ktutil''' to encrypt your Service Password into a keytab file:

''interactive way:''
<pre>
ktutil
ktutil: addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e rc4-hmac
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
ktutil: addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e aes256-cts
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
ktutil: wkt xy123.keytab
ktuitl: quit
</pre>

''non-interactive way:''
<pre>
echo -e "addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e rc4-hmac\n<your_servicepasword>\n
addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e aes256-cts\n<your_servicepasword>\nwkt xy123.keytab" | ktutil
</pre>

With this keytab, you can fetch a kerberos ticket without an interactive password:
<pre>
kinit -k -t xy123.keytab hd_xy123
</pre>
<br>

= <b> Troubleshooting </b> =

== incorrect mount option ==

When you try to mount SDS@hd you get the following error message:
<pre>
mount.nfs4: mount(2): Invalid argument
mount.nfs4: an incorrect mount option was specified
</pre>

This means the service rpc-gssd.service is not yet running. Please try to start the service via <pre>systemctl restart rpc-gssd.service</pre>

== permission denied while mounting when SELinux is enabled ==

When trying to mount, you get the following error message:
<pre>
mount.nfs4: mount(2): Permission denied
mount.nfs4: trying text-based options 'sec=krb5,vers=4.1,addr=x.x.x.x,clientaddr=a.b.c.d'
mount.nfs4: mount(2): Permission denied
mount.nfs4: access denied by server while mounting lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02
</pre>

You should check your systemlogs for entries like:
<pre>
Aug 07 13:09:53 nc-docker-aio-don setroubleshoot[510523]: SELinux is preventing /usr/sbin/rpc.gssd from open access on the file /etc/krb5.keytab. For complete SELinux messages run: sealert -l XXXXXX
</pre>

or similar. To solve the issue you have to add a SELinux rule to allow access to /etc/krb5.keytab (or disable SElinux completely).

SDS@hd/Access/NFS

2024-01-10T10:56:50Z

S Siebler: /* Prerequisites */

SDS@hd/Access

2023-11-20T11:46:50Z

S Siebler:

= Access Protocols =
* [[SDS@hd/Access/SFTP|SFTP/SSHFS (Win,Mac,Linux)]]
* [[SDS@hd/Access/CIFS|SMB/CIFS (Win,Mac,Linux)]]
* [[SDS@hd/Access/NFS|NFSv4 (Linux)]]
* [[SDS@hd/Access/WEBDAV|WebDAV (Win,Mac,Linux)]]




= Access from bwForCluster Helix =
On bwForCluster Helix is possible to directly access your storage space under /mnt/sds-hd/ on all login and compute nodes.

SDS@hd/Access/NFS

2023-11-15T08:59:01Z

S Siebler: /* mount a nfs share */

= <b> Prerequisites </b> =

* '''Attention:''' To access data served by SDS@hd, You need a '''''Service Password'''''. See details [[SDS@hd/Registration]].

* Additionally the access to SDS@hd is currently only available inside the [https://www.belwue.de/netz/netz0.html belwue-Network]. This means you have to use the VPN Service of your HomeOrganization, if you want to access SDS@hd from outside the bwHPC-Clusters (e.g. via [https://www.eduroam.org/where/ eduroam] or from your personal Laptop)

* The access via nfs protocol is machine-based, which means '''new nfs-Clients have to be registered''' on SDS@hd. During this registration each machine gets a keytab file, which allows mounting SDS@hd.

* Currently you have to [mailto:sds-hd-support@urz.uni-heidelberg.de?subject=SDS@hd%20nfs-Client%20Registration send an email] for Clientregistration to SDS@hd Team with the following information:
** hostname of the new nfs-Client
** IP address
** short description
** location
** acronym of the Speichervorhaben which should be available on this machine

= <b> Using NFSv4 for UNIX client </b> =

The authentication for data access via NFSv4 is performed using Kerberostickets. This requires a functioning Kerberos environment on the client!

{{:SDS@hd/Access/Kerberos}}

After configuring kerberos, you have to install nfs packages in your system, and enable kerberized NFSv4. The exact names of the packages depending on you linux distribution (see examples below).

''Example RedHat/CentOS''
<pre>
> yum install nfs-utils nfs4-acl-tools

/etc/sysconfig/nfs:
NEED_IDMAPD=yes
NEED_GSSD=yes
</pre>

''Example debian/ubuntu''
<pre>
> apt install nfs-common nfs4-acl-tools nfs-server

/etc/default/nfs-common:
NEED_IDMAPD=yes
NEED_GSSD=yes
</pre>
On ubuntu server: nfs-kernel-server

{{:SDS@hd/Access/ID-Mapping}}

To enable the ID-Mapping for NFSv4 mounts change the file ''/etc/idmapd.conf'' with the following lines:
<pre>
in /etc/idmapd.conf:
[General]
Domain = urz.uni-heidelberg.de
Local-Realms = BWSERVICES.UNI-HEIDELBERG.DE
</pre>

== mount a nfs share ==
The usual restrictions for mounting drives under Linux apply. Usually this can only be done by the superuser "root". For detailed information, please contact the system administrator of your system.

After successfull configuration (s. 2.1) you can mount your SDS@hd share with the following commands:
<pre>
> mkdir <mountpoint>
> mount -t nfs4 -o sec=krb5,vers=4.1 lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02/ <mountpoint>
</pre>

To enable the mounting after a restart, you have to add the following line to the file "/etc/fstab"
<pre>
lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02/ <mountpoint> nfs4 sec=krb5,vers=4.1 0 0
</pre>

=== AutoFS Setup ===

Instead of the fstab-entry you can also use the automounter "autofs".

* RedHat/CentOS:
<pre>
$ yum install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

* debian/ubuntu:
<pre>
$ apt install autofs
$ systemctl enable autofs
$ systemctl start autofs
</pre>

Afterwards you configure the SDS@hd Speichervorhaben in a new map file:
<pre>
$ cat /etc/auto.sds-hd
sds-hd -fstype=nfs4,rw,sec=krb5,vers=4.1,nosuid,nodev lsdf02.urz.uni-heidelberg.de:/gpfs/lsdf02
....
</pre>

You have to include the new map into the auto.master file, e.g.:
<pre>
$ cat /etc/auto.master
[...]
/mnt /etc/auto.sds-hd
[...]
</pre>

To display all available SDS@hd shares on this machine to the users, you should enable "browser_mode":
<pre>
$ cat /etc/autofs.conf
[...]
# to display all available SDS-hd shares on this to the users
browse_mode=yes
[...]
</pre>
otherwise each share-folder will only be visible after a user has mounted.

After changing the configuration, you should restart the autofs daemon, e.g.:
<pre>
$ systemctl restart autofs
</pre>

Of course you can adopt all other autofs options, like timeouts, etc. to the specific needs of your environment or use any other method for dynamically mounting the shares.

== access your data ==
'''Attention!''' The access can not be done as root user, because root uses the Kerberosticket of the machine, which does not have data access!

To access your data on SDS@hd you have to fetch a valid kerberos ticket with your SDS@hd user and Servicepassword:
<pre>
> kinit hd_xy123
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
</pre>
You can check afterwards your kerberos ticket with:
<pre>
> klist
Ticket cache: FILE:/tmp/krb5cc_1000
Default principal: hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE

Valid starting Expires Service principal
20.09.2017 04:00:01 21.09.2017 04:00:01 krbtgt/BWSERVICES.UNI-HEIDELBERG.DE@BWSERVICES.UNI-HEIDELBERG.DE
renew until 29.09.2017 13:38:49
</pre>

Afterwards you should be able to access the mountpoint, which contain all Speichervorhaben exported to your machine:
<pre>
> ls <mountpoint>
sd16j007 sd17c010 sd17d005
</pre>

== renew a kerberos ticket ==
Because a kerberos ticket has a limited lifetime (default: 10 hours, maximum 24 hours) for security reasons, you have to renew your ticket before it expires to prevent access loss.
<pre>
> kinit -R
</pre>

This renewal could only be done for maximum time of 10 Days and as long as the current kerberos ticket is still valid. For renewal of an expired ticket, you have to use again your Servicepassword.

== destroy kerberos ticket ==
Even if kerberos tickets are only valid for a limited period of time, a ticket should be destroyed as soon as access is no longer needed to prevent misuse on multi-user systems:
<pre>kdestroy</pre>

== automated kerberos tickets ==
<span style="color:red"><strong>'''Attention!''' Keep this generated Keytab safe and use it only in trusted environments!</strong></span>

If your workflow needs a permanent access to SDS@hd for longer than 10 Days, you can use '''ktutil''' to encrypt your Service Password into a keytab file:

''interactive way:''
<pre>
ktutil
ktutil: addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e rc4-hmac
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
ktutil: addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e aes256-cts
Password for hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE:
ktutil: wkt xy123.keytab
ktuitl: quit
</pre>

''non-interactive way:''
<pre>
echo -e "addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e rc4-hmac\n<your_servicepasword>\n
addent -password -p hd_xy123@BWSERVICES.UNI-HEIDELBERG.DE -k 1 -e aes256-cts\n<your_servicepasword>\nwkt xy123.keytab" | ktutil
</pre>

With this keytab, you can fetch a kerberos ticket without an interactive password:
<pre>
kinit -k -t xy123.keytab hd_xy123
</pre>
<br>

SDS@hd

2023-09-28T06:54:16Z

S Siebler: dienstöffnung

[[File:sds-hd-logo.png|350px]]

SDS@hd is a central service for securely storing scientific data (Scientific Data Storage). The service is provided as a state service to researchers of higher education institutions of Baden-Württemberg. It is intended to be used for data that is frequently accessed ('hot data').

{| style=" background:#FEF4AB; width:100%;"
| style="padding:8px; background:#FFE856; font-size:120%; font-weight:bold; text-align:left" | News
|-
|
* November 2022: No special entitlement is needed anymore to participate in an existing storage project.
* September 2023: Service has now been opened for DFN AAI & eduGAIN federation members to participate on existing storage projects.
|}

{| style=" background:#eeeefe; width:100%;"
| style="padding:8px; background:#dedefe; font-size:120%; font-weight:bold; text-align:left" | Training & Support
|-
|
* [https://www.urz.uni-heidelberg.de/en/service-catalogue/storage/sdshd-scientific-data-storage Service description & FAQ]
* [mailto:sds-hd-support@urz.uni-heidelberg.de Submit a Ticket]
|}

{| style=" background:#ffeaef; width:100%;"
| style="padding:8px; background:#f5dfdf; font-size:120%; font-weight:bold; text-align:left" | User Documentation
|-
|
* [[SDS@hd/Registration|Registration]]
* [[SDS@hd/Access|Access]]
* [[SDS@hd/Security|Data Security]]
* Visualization: [https://www.bwvisu.de/ bwVisu]
|}

{| style=" background:#e6e9eb; width:100%;"
| style="padding:8px; background:#d1dadf; font-size:120%; font-weight:bold; text-align:left" | Storage Funding
|-
|
* Please [[SDS@hd/Acknowledgement|acknowledge]] SDS@hd in your publications.

SDS@hd/Registration

2022-12-14T14:05:05Z

S Siebler: update entitlement requirements

Granting access and issuing a user account for '''''SDS@hd''''' requires the registration at the bwServices website [https://bwservices.uni-heidelberg.de/ https://bwservices.uni-heidelberg.de] (step 2).

= Storageproject (Speichervorhaben) =

Before you can use SDS@hd you will have to register for a new "Speichervorhaben" or contribute to an already existing project on [https://sds-hd.urz.uni-heidelberg.de/management SDS@hd Managementtool].

== Register a new "SV" ==

Typically done only by the leader of a scientific work group or the senior scientist of a research group/collaboration.
Any amount of co-workers can join your SV without having to register another project.

Your own high school or university has to grant you the permission to start an own SDS@hd Storageproject ("SDS@hd SV entitlement").

Each university has their own procedure.

For more details about the SDS@hd entitlements you can take a look at the [https://urz.uni-heidelberg.de/en/sds-hd SDS@hd Website]


If you register your own SV, you will be:
# held accountable for the co-workers in the SV
# asked to provide information for the two reports required by the DFG for their funding of SDS@hd
# likely asked for a contribution to a future DFG grant proposal for an extension of the storage system in your area of research ("wissenschaftliches Beiblatt")

== Become Coworker of an "SV"==

Your advisor (the "SV" responsible") will provide you with the following data on the SV:
* acronym
* password

To become coworker of an ''SV'', please login at
* [https://sds-hd.urz.uni-heidelberg.de/management/index.php?mode=mitarbeit SDS@hd Managementtool] and provide acronym and password.

You will be assigned to the 'SV' as a member.

After submitting the request you will receive an email about the further steps.
The SV owner and any managers will be notified automatically.

= Personal registration for SDS@hd =

After step 1 you have to register your personal account on the storage system and set a service password.
Please visit:
* [https://bwservices.uni-heidelberg.de/ https://bwservices.uni-heidelberg.de]
*# Select your home organization from the list and click '''Proceed'''
*# You will be directed to the ''Identity Provider'' of your home organisation
*# Enter your home-organisational user ID / username and your home-organisational password and click '''Login''' button
*# You will be redirected back to the registration website [https://bwservices.uni-heidelberg.de/ https://bwservices.uni-heidelberg.de/]
*# <div>Select unter '''The following services are available''' the service '''SDS@hd - Scientific Data Storage'''
*# Click '''Register'''
*# Finally, set a service password for authentication on SDS@hd

[[File:Sds_bwservices_servicepassword.png]]
<br/>

== Changing Password ==

At any time, you can set a new SDS@hd service password via the registration website [https://bwservices.uni-heidelberg.de/ https://bwservices.uni-heidelberg.de] by carrying out the following steps:
# visit [https://bwservices.uni-heidelberg.de/ https://bwservices.uni-heidelberg.de] and select your home organization
# authenticate yourself via your home-organizational user id / username and your home-organizational password
# find on the left side '''SDS@hd''' and select '''Set Service Password'''
# set new service password, repeat it and click '''Save''' button.
# the page answers e.g. "Das Passwort wurde bei dem Dienst geändert" ("password has been changed")

Helix/Filesystems

2022-12-07T15:59:25Z

S Siebler: /* Workspaces */

== Overview ==

The cluster storage system provides a large parallel file system based on [https://www.ibm.com/support/knowledgecenter/STXKQY/ibmspectrumscale_welcome.html IBM Spectrum Scale]
for $HOME, for workspaces, and for temporary storage via the $TMPDIR environment variable.

{| class="wikitable"
|-
!style="width:16%" |
!style="width:28%"| $HOME
!style="width:28%"| Workspaces
!style="width:28%"| $TMPDIR
|-
!scope="column" | Visibility
| global
| global
| local
|-
!scope="column" | Lifetime
| permanent
| workspace lifetime
| batch job walltime
|-
!scope="column" | Quotas
| 200 GB
| 10 TB
| none
|-
!scope="column" | Backup
| no
| no
| no
|}

* global: all nodes access the same file system.
* local: each node has its own temporary file space.
* permanent: files are stored permanently.
* workspace lifetime: files are removed at end of workspace lifetime.
* batch job walltime: files are removed at end of the batch job.

== $HOME ==

Home directories are meant for permanent storage of files that are kept being used like source codes, configuration files, executable programs. There is currently no backup for the home directory. The disk space per user is limited to 200 GB. The used disk space is displayed with the command:

<code>homequotainfo</code>

== Workspaces ==

Workspace tools can be used to get temporary space for larger amounts of data necessary for or produced by running jobs. To create a workspace you need to supply a name for the workspace and a lifetime in days. The maximum lifetime is 30 days. It is possible to extend the lifetime 10 times.

{| class="wikitable"
|-
!style="width:30%" | Command
!style="width:70%"| Action
|-
|ws_allocate -r 7 -m <email> foo 10
|Allocate a workspace named foo for 10 days and set a email reminder 7 days before expiring.
'''Important:''' You should add your email address and a relative date for notification!
|-
|ws_list -a
|List all your workspaces.
|-
|ws_find foo
|Get absolute path of workspace foo.
|-
|ws_extend foo 5
|Extend lifetime of workspace foo by 5 days from now.
|-
|ws_release foo
|Manually erase your workspace foo.
|-
|ws_send_ical -m <email> foo
|sending a .ical calendar entry for reminding workspace expiring
|-
|ws_share share/unshare/unshare-all <workspacename> <username> [username2]
|ws_share allows to share an existing workspace with other users.
|-
|}

If you plan to produce or copy large amounts of data in workspaces, please check the availability. The used and free disk space on the workspace filesystem is displayed with the command:

<code>workquotainfo</code>

==== Restoring expired Workspaces ====
At expiration time your workspace will be moved to a special, hidden directory. For a short period of time you can still restore your data into a valid workspace. For that, use
<pre>
ws_restore -l
</pre>
to get a list of your expired workspaces, and then restore them into an '''existing, active workspace''' <code>restored_ws</code>:
<pre>
ws_restore <deleted_workspacename> restored_ws
</pre>
NOTE: the expired workspace has to be specified using the full name as listed by <code>ws_restore -l</code>, including username prefix and timestamp suffix (otherwise, it cannot be uniquely identified).
The target workspace, on the other hand, must be given with just its short name as listed by <code>ws_list</code>, without the username prefix.

NOTE: <code>ws_restore</code> can only work on the same filesystem! So you have to ensure that the new workspace allocated with <code>ws_allocate</code> is placed on the same filesystem as the expired workspace. Therefor you can use <code>-F <filesystem></code> flag if needed.

==== Linking workspaces in Home ====
It might be valuable to have links to personal workspaces within a certain directory, e.g., the user home directory. The command
<pre>
ws_register <DIR>
</pre>
will create and manage links to all personal workspaces within in the directory <DIR>. Calling this command will do the following:

* The directory <DIR> will be created if necessary
* Links to all personal workspaces will be managed:
** Creates links to all available workspaces if not already present
** Removes links to released workspaces

==== Sharing Workspaces ====
To simplify sharing of workspaces you can use the command
<pre>
ws_share
</pre>

This tool setup the needed ACLs on the workspace to allow read-only access to the specified users.

If you need more specific permissions, you can also setup the ACLs on your own, like described here:
https://wiki.bwhpc.de/e/Workspace#Sharing_Workspace_Data_within_your_Workgroup

==== Troubleshooting ====
If you are getting the error:
<pre>
Error: could not create workspace directory!
</pre>
you should check the localesetting of your ssh client. Some clients (e.g. the one from MacOSX) set values that are not valid. You should overwrite LC_CTYPE and set it to a valid locale value like:
<pre>
export LC_CTYPE=de_DE.UTF-8
</pre>
To make this change persistent you can add this line also to your <code>.bashrc</code> file.

A list of valid locales can be retrieved via
<pre>
locale -a
</pre>

If you are getting the error:
<pre>
Error: only root can do this <workspacename>
</pre>
ensure that you are using the FULL workspacename, including your account_prefix, e.g. "hd_ab123-workspacename"

== $TMPDIR ==

The variable $TMPDIR provides file space for temporary data of running jobs.
Each node has its own $TMPDIR. 
The data in $TMPDIR become unavailable as soon as the job has finished.

== Access to SDS@hd ==

It is possible to access your storage space on [http://sds-hd.urz.uni-heidelberg.de SDS@hd] directly on the bwForCluster Helix in /mnt/sds-hd/ on all login and compute nodes.

Helix/Filesystems

2022-12-07T15:55:22Z

S Siebler: /* Workspaces */

== Overview ==

The cluster storage system provides a large parallel file system based on [https://www.ibm.com/support/knowledgecenter/STXKQY/ibmspectrumscale_welcome.html IBM Spectrum Scale]
for $HOME, for workspaces, and for temporary storage via the $TMPDIR environment variable.

{| class="wikitable"
|-
!style="width:16%" |
!style="width:28%"| $HOME
!style="width:28%"| Workspaces
!style="width:28%"| $TMPDIR
|-
!scope="column" | Visibility
| global
| global
| local
|-
!scope="column" | Lifetime
| permanent
| workspace lifetime
| batch job walltime
|-
!scope="column" | Quotas
| 200 GB
| 10 TB
| none
|-
!scope="column" | Backup
| no
| no
| no
|}

* global: all nodes access the same file system.
* local: each node has its own temporary file space.
* permanent: files are stored permanently.
* workspace lifetime: files are removed at end of workspace lifetime.
* batch job walltime: files are removed at end of the batch job.

== $HOME ==

Home directories are meant for permanent storage of files that are kept being used like source codes, configuration files, executable programs. There is currently no backup for the home directory. The disk space per user is limited to 200 GB. The used disk space is displayed with the command:

<code>homequotainfo</code>

== Workspaces ==

Workspace tools can be used to get temporary space for larger amounts of data necessary for or produced by running jobs. To create a workspace you need to supply a name for the workspace and a lifetime in days. The maximum lifetime is 30 days. It is possible to extend the lifetime 10 times.

{| class="wikitable"
|-
!style="width:30%" | Command
!style="width:70%"| Action
|-
|ws_allocate -r 7 -m <email> foo 10
|Allocate a workspace named foo for 10 days and set a email reminder 7 days before expiring.
'''Important:''' You should add your email address and a relative date for notification!
|-
|ws_list -a
|List all your workspaces.
|-
|ws_find foo
|Get absolute path of workspace foo.
|-
|ws_extend foo 5
|Extend lifetime of workspace foo by 5 days from now.
|-
|ws_release foo
|Manually erase your workspace foo.
|-
|ws_send_ical -m <email> foo
|sending a .ical calendar entry for reminding workspace expiring
|-
|ws_share share/unshare/unshare-all <workspacename> <username> [username2]
|ws_share allows to share an existing workspace with other users.
|-
|}

If you plan to produce or copy large amounts of data in workspaces, please check the availability. The used and free disk space on the workspace filesystem is displayed with the command:

<code>workquotainfo</code>

==== Restoring expired Workspaces ====
At expiration time your workspace will be moved to a special, hidden directory. For a short period of time you can still restore your data into a valid workspace. For that, use
<pre>
ws_restore -l
</pre>
to get a list of your expired workspaces, and then restore them into an '''existing, active workspace''' <code>restored_ws</code>:
<pre>
ws_restore <deleted_workspacename> restored_ws
</pre>
NOTE: the expired workspace has to be specified using the full name as listed by <code>ws_restore -l</code>, including username prefix and timestamp suffix (otherwise, it cannot be uniquely identified).
The target workspace, on the other hand, must be given with just its short name as listed by <code>ws_list</code>, without the username prefix.

NOTE: <code>ws_restore</code> can only work on the same filesystem! So you have to ensure that the new workspace allocated with <code>ws_allocate</code> is placed on the same filesystem as the expired workspace. Therefor you can use <code>-F <filesystem></code> flag if needed.

==== Linking workspaces in Home ====
It might be valuable to have links to personal workspaces within a certain directory, e.g., the user home directory. The command
<pre>
ws_register <DIR>
</pre>
will create and manage links to all personal workspaces within in the directory <DIR>. Calling this command will do the following:

* The directory <DIR> will be created if necessary
* Links to all personal workspaces will be managed:
** Creates links to all available workspaces if not already present
** Removes links to released workspaces

==== Troubleshooting ====
If you are getting the error:
<pre>
Error: could not create workspace directory!
</pre>
you should check the localesetting of your ssh client. Some clients (e.g. the one from MacOSX) set values that are not valid. You should overwrite LC_CTYPE and set it to a valid locale value like:
<pre>
export LC_CTYPE=de_DE.UTF-8
</pre>
To make this change persistent you can add this line also to your <code>.bashrc</code> file.

A list of valid locales can be retrieved via
<pre>
locale -a
</pre>

If you are getting the error:
<pre>
Error: only root can do this <workspacename>
</pre>
ensure that you are using the FULL workspacename, including your account_prefix, e.g. "hd_ab123-workspacename"

== $TMPDIR ==

The variable $TMPDIR provides file space for temporary data of running jobs.
Each node has its own $TMPDIR. 
The data in $TMPDIR become unavailable as soon as the job has finished.

== Access to SDS@hd ==

It is possible to access your storage space on [http://sds-hd.urz.uni-heidelberg.de SDS@hd] directly on the bwForCluster Helix in /mnt/sds-hd/ on all login and compute nodes.

Helix/Filesystems

2022-10-12T08:08:05Z

S Siebler: /* Troubleshooting */

== Overview ==

The cluster storage system provides a large parallel file system based on [https://www.ibm.com/support/knowledgecenter/STXKQY/ibmspectrumscale_welcome.html IBM Spectrum Scale]
for $HOME, for workspaces, and for temporary storage via the $TMPDIR environment variable.

{| class="wikitable"
|-
!style="width:16%" |
!style="width:28%"| $HOME
!style="width:28%"| Workspaces
!style="width:28%"| $TMPDIR
|-
!scope="column" | Visibility
| global
| global
| local
|-
!scope="column" | Lifetime
| permanent
| workspace lifetime
| batch job walltime
|-
!scope="column" | Quotas
| 200 GB
| 10 TB
| none
|-
!scope="column" | Backup
| no
| no
| no
|}

* global: all nodes access the same file system.
* local: each node has its own temporary file space.
* permanent: files are stored permanently.
* workspace lifetime: files are removed at end of workspace lifetime.
* batch job walltime: files are removed at end of the batch job.

== $HOME ==

Home directories are meant for permanent storage of files that are kept being used like source codes, configuration files, executable programs. There is currently no backup for the home directory. The disk space per user is limited to 200 GB. The used disk space is displayed with the command:

<code>homequotainfo</code>

== Workspaces ==

Workspace tools can be used to get temporary space for larger amounts of data necessary for or produced by running jobs. To create a workspace you need to supply a name for the workspace and a lifetime in days. The maximum lifetime is 30 days. It is possible to extend the lifetime 10 times.

{| class="wikitable"
|-
!style="width:30%" | Command
!style="width:70%"| Action
|-
|ws_allocate -r 7 -m <email> foo 10
|Allocate a workspace named foo for 10 days and set a email reminder 7 days before expiring.
'''Important:''' You should add your email address and a relative date for notification!
|-
|ws_list -a
|List all your workspaces.
|-
|ws_find foo
|Get absolute path of workspace foo.
|-
|ws_extend foo 5
|Extend lifetime of workspace foo by 5 days from now.
|-
|ws_release foo
|Manually erase your workspace foo.
|-
|ws_send_ical -m <email> foo
|sending a .ical calendar entry for reminding workspace expiring
|-
|}

If you plan to produce or copy large amounts of data in workspaces, please check the availability. The used and free disk space on the workspace filesystem is displayed with the command:

<code>workquotainfo</code>

==== Restoring expired Workspaces ====
At expiration time your workspace will be moved to a special, hidden directory. For a short period of time you can still restore your data into a valid workspace. For that, use
<pre>
ws_restore -l
</pre>
to get a list of your expired workspaces, and then restore them into an '''existing, active workspace''' <code>restored_ws</code>:
<pre>
ws_restore <deleted_workspacename> restored_ws
</pre>
NOTE: the expired workspace has to be specified using the full name as listed by <code>ws_restore -l</code>, including username prefix and timestamp suffix (otherwise, it cannot be uniquely identified).
The target workspace, on the other hand, must be given with just its short name as listed by <code>ws_list</code>, without the username prefix.

NOTE: <code>ws_restore</code> can only work on the same filesystem! So you have to ensure that the new workspace allocated with <code>ws_allocate</code> is placed on the same filesystem as the expired workspace. Therefor you can use <code>-F <filesystem></code> flag if needed.

==== Linking workspaces in Home ====
It might be valuable to have links to personal workspaces within a certain directory, e.g., the user home directory. The command
<pre>
ws_register <DIR>
</pre>
will create and manage links to all personal workspaces within in the directory <DIR>. Calling this command will do the following:

* The directory <DIR> will be created if necessary
* Links to all personal workspaces will be managed:
** Creates links to all available workspaces if not already present
** Removes links to released workspaces

==== Troubleshooting ====
If you are getting the error:
<pre>
Error: could not create workspace directory!
</pre>
you should check the localesetting of your ssh client. Some clients (e.g. the one from MacOSX) set values that are not valid. You should overwrite LC_CTYPE and set it to a valid locale value like:
<pre>
export LC_CTYPE=de_DE.UTF-8
</pre>
To make this change persistent you can add this line also to your <code>.bashrc</code> file.

A list of valid locales can be retrieved via
<pre>
locale -a
</pre>

If you are getting the error:
<pre>
Error: only root can do this <workspacename>
</pre>
ensure that you are using the FULL workspacename, including your account_prefix, e.g. "hd_ab123-workspacename"

== $TMPDIR ==

The variable $TMPDIR provides file space for temporary data of running jobs.
Each node has its own $TMPDIR. 
The data in $TMPDIR become unavailable as soon as the job has finished.

== Access to SDS@hd ==

It is possible to access your storage space on [http://sds-hd.urz.uni-heidelberg.de SDS@hd] directly on the bwForCluster Helix in /mnt/sds-hd/ on all login and compute nodes.

Registration/SSH

2022-03-30T13:28:13Z

S Siebler: /* Revoke/Delete SSH Key */

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
This process is only necessary for the bwUniCluster and the bwForCluster MLS&WISO.
On the other clusters, SSH keys can still be copied to the <code>authorized_keys</code> file.
|}

= Registering SSH Keys with your Cluster =

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
Interactive SSH Keys are not valid all the time, but only for one hour after the last 2-factor login.
They have to be "unlocked" by entering the OTP and service password.
|}

'''SSH Keys''' are a mechanism for logging into a computer system without having to enter a password. Instead of authenticating yourself with something you know (a password), you prove your identity by showing the server something you have (a cryptographic key).

The usual process is the following:

* The user generates a pair of SSH Keys, a private key and a public key, on their local system. The private key never leaves the local system.

* The user then logs into the remote system using the remote system password and adds the public key to a file called ~/.ssh/authorized_keys .

* All following logins will no longer require the entry of the remote system password because the local system can prove to the remote system that it has a private key matching the public key on file.

While SSH Keys have many advantages, the concept also has '''a number of issues''' which make it hard to handle them securely:

* The private key on the local system is supposed to be protected by a strong passphrase. There is no possibility for the server to check if this is the case. Many users do not use a strong passphrase or do not use any passphrase at all. If such a private key is stolen, an attacker can immediately use it to access the remote system.

* There is no concept of validity. Users are not forced to regularly generate new SSH Key pairs and replace the old ones. Often the same key pair is used for many years and the users have no overview of how many systems they have stored their SSH Keys on.

* SSH Keys can be restricted so they can only be used to execute specific commands on the server, or to log in from specified IP addresses. Most users do not do this.

To fix these issues '''it is no longer possible to self-manage your SSH Keys by adding them to the ~/.ssh/authorized_keys file''' on bwUniCluster/bwForCluster.
SSH Keys have to be managed through bwIDM/bwServces instead.
Existing authorized_keys files are ignored.

== Minimum requirements for SSH Keys ==

Algorithms and Key sizes:

* 2048 bits or more for RSA
* 521 bits for ECDSA
* 256 Bits (Default) for ED25519

ECDSA-SK and ED25519-SK keys (for use with U2F Hardware Tokens) cannot be used yet.

'''Please set a strong passphrase for your private keys.'''

= Adding a new SSH Key =

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
* Newly added keys are valid for three months. After that, they are revoked and placed on a "revocation list" so that they cannot be reused.
* Copy only the contents of your public ssh key file to bwIDM/bwServices. The file ends with <code>.pub</code> ( e.g. <code>~/.ssh/<filename>.pub</code>).
|}

'''SSH keys''' are generally managed via the '''My SSH Pubkeys''' menu entry on the registration pages for the clusters.
Here you can add and revoke SSH keys. To add a ssh key, please follow these steps:

1. '''Select the cluster''' for which you want to create a second factor:</br> → [https://login.bwidm.de/user/ssh-keys.xhtml '''bwUniCluster 2.0''']</br> → [https://bwservices.uni-heidelberg.de/user/ssh-keys.xhtml '''bwForCluster MLS&WISO''']
[[File:BwIDM-twofa.png|center|600px|thumb|My SSH Pubkeys.]]

3. Click the '''Add SSH Key''' or '''SSH Key Hochladen''' button.
[[File:Bwunicluster 2.0 access ssh keys empty.png|center|400px|thumb|Add new SSH key.]]

4. A new window will appear.
Enter a name for the key and paste your SSH public key (file <code>~/.ssh/<filename>.pub</code>) into the box labelled "SSH Key:".
Click on the button labelled '''Add''' or '''Hinzufügen'''.
[[File:Ssh-key.png|center|600px|thumb|Add new SSH key.]]

5. If everything worked fine your new key will show up in the user interface:
[[File:Ssh-success.png|center|800px|thumb|New SSH key added.]]

Once you have added SSH keys to the system, you can bind them to one or more services to use either for interactive logins ('''Interactive key''') or for automatic logins ('''Command key''').

== Registering an Interactive Key ==

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
Interactive SSH Keys are not valid all the time, but only for one hour after the last 2-factor login.
They have to be "unlocked" by entering the OTP and service password.
|}

'''Interactive Keys''' can be used to log into a system for interactive use.
Perform the following steps to register an interactive key:

1. [[Registration/SSH#Adding_a_new_SSH_Key|'''Add a new interactive SSH key''']] if you have not already done so.

2. Select '''Registered services/Registrierte Dienste''' from the top menu and click '''Set SSH Key/SSH Key setzen''' for the cluster for which you want to use the SSH key.
[[File:BwIDM-registered.png|center|600px|thumb|Select Cluster for which you want to use the SSH key.]]

3. The upper block displays the SSH keys currently registered for the service.
The bottom block displays all the public SSH keys associated with your account.
Find the SSH key you want to use and click '''Add/Hinzufügen'''.
[[File:Ssh-service-int.png|center|800px|thumb|Add SSH key to service.]]

4. A new window appears.
Select '''Interactive''' as the usage type, enter an optional comment and click '''Add/Hinzufügen'''.
[[File:Ssh-int.png|center|600px|thumb|Add interactive SSH key to service.]]

5. Your SSH key is now registered for interactive use with this service.
[[File:Ssh-service.png|center|800px|thumb|SSH key is now registered for interactive use.]]

== Registering a Command Key ==

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
SSH command keys are always valid and do not need to be unlocked with a 2-factor login.
This makes these keys extremely valuable to a potential attacker and poses a security risk.
Therefore, additional restrictions apply to these keys:
* They must be limited to a single command to be executed.
* They must be limited to a single IP address (e.g., the workflow server) or a small number of IP addresses (e.g., the institution's subnet).
* They must be reviewed and approved by a cluster administrator before they can be used.
* Validity is reduced to one month.
|}

'''Command Keys''' can be used for automatic workflows.
Perform the following steps to register a "Command key":

1. [[Registration/SSH#Adding_a_new_SSH_Key|'''Add a new "command SSH key"''']] if you have not already done so.

2. Select '''Registered services/Registrierte Dienste''' from the top menu and click '''Set SSH Key/SSH Key setzen''' for the cluster for which you want to use the SSH key.
[[File:BwIDM-registered.png|center|600px|thumb|Select Cluster for which you want to use the SSH key.]]

3. The upper block displays the SSH keys currently registered for the service.
The bottom block displays all the public SSH keys associated with your account.
Find the SSH key you want to use and click '''Add/Hinzufügen'''.
[[File:Ssh-service-com.png|center|800px|thumb|Add SSH key to service.]]

4. A new window appears.
Select '''Command''' as the usage type.
Type the full command with the full path, including all parameters, in the '''Command''' text box.
Specify a network address, list, or range in the '''From''' text field (see [https://man.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man8/sshd.8#from=_pattern-list_ man 8 sshd] for more info).
Please also provide a comment to speed up the approval process.
Click '''Add/Hinzufügen'''.
{| class="wikitable"
! | Example
|-
| If you want to register a command key to be able to transfer data automatically, please use the following string as in the '''Command''' text field (please verify the path on the cluster first):
<pre>
/usr[/local]/bin/rrsync -ro / -rw /
</pre>
|}
[[File:Ssh-com.png|center|600px|thumb|Add interactive SSH key to service.]]

5. After the key has been added, it will be marked as '''Pending''':
You will receive an e-mail as soon as the key has been approved and can be used.
[[File:Ssh-service.png|center|800px|thumb|SSH key is now registered for interactive use.]]

== Revoke/Delete SSH Key ==

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
Revoked keys are locked and can no longer be used.
|}

'''SSH keys''' are generally managed via the '''My SSH Pubkeys''' menu entry on the registration pages for the clusters.
Here you can add and revoke SSH keys. To revoke/delete a ssh key, please follow these steps:

1. '''Select the cluster''' for which you want to create a second factor:</br> → [https://login.bwidm.de/user/ssh-keys.xhtml '''bwUniCluster 2.0''']</br> → [https://bwservices.uni-heidelberg.de/user/ssh-keys.xhtml '''bwForCluster MLS&WISO''']
[[File:BwIDM-twofa.png|center|600px|thumb|My SSH Pubkeys.]]

2. Click '''REVOKE/ZURÜCKZIEHEN''' next to the SSH key you want to revoke.
[[File:Ssh-success.png|center|800px|thumb|Revoke SSH key.]]

Registration/SSH

2022-03-30T13:27:20Z

S Siebler: /* Adding a new SSH Key */

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
This process is only necessary for the bwUniCluster and the bwForCluster MLS&WISO.
On the other clusters, SSH keys can still be copied to the <code>authorized_keys</code> file.
|}

= Registering SSH Keys with your Cluster =

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
Interactive SSH Keys are not valid all the time, but only for one hour after the last 2-factor login.
They have to be "unlocked" by entering the OTP and service password.
|}

'''SSH Keys''' are a mechanism for logging into a computer system without having to enter a password. Instead of authenticating yourself with something you know (a password), you prove your identity by showing the server something you have (a cryptographic key).

The usual process is the following:

* The user generates a pair of SSH Keys, a private key and a public key, on their local system. The private key never leaves the local system.

* The user then logs into the remote system using the remote system password and adds the public key to a file called ~/.ssh/authorized_keys .

* All following logins will no longer require the entry of the remote system password because the local system can prove to the remote system that it has a private key matching the public key on file.

While SSH Keys have many advantages, the concept also has '''a number of issues''' which make it hard to handle them securely:

* The private key on the local system is supposed to be protected by a strong passphrase. There is no possibility for the server to check if this is the case. Many users do not use a strong passphrase or do not use any passphrase at all. If such a private key is stolen, an attacker can immediately use it to access the remote system.

* There is no concept of validity. Users are not forced to regularly generate new SSH Key pairs and replace the old ones. Often the same key pair is used for many years and the users have no overview of how many systems they have stored their SSH Keys on.

* SSH Keys can be restricted so they can only be used to execute specific commands on the server, or to log in from specified IP addresses. Most users do not do this.

To fix these issues '''it is no longer possible to self-manage your SSH Keys by adding them to the ~/.ssh/authorized_keys file''' on bwUniCluster/bwForCluster.
SSH Keys have to be managed through bwIDM/bwServces instead.
Existing authorized_keys files are ignored.

== Minimum requirements for SSH Keys ==

Algorithms and Key sizes:

* 2048 bits or more for RSA
* 521 bits for ECDSA
* 256 Bits (Default) for ED25519

ECDSA-SK and ED25519-SK keys (for use with U2F Hardware Tokens) cannot be used yet.

'''Please set a strong passphrase for your private keys.'''

= Adding a new SSH Key =

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
* Newly added keys are valid for three months. After that, they are revoked and placed on a "revocation list" so that they cannot be reused.
* Copy only the contents of your public ssh key file to bwIDM/bwServices. The file ends with <code>.pub</code> ( e.g. <code>~/.ssh/<filename>.pub</code>).
|}

'''SSH keys''' are generally managed via the '''My SSH Pubkeys''' menu entry on the registration pages for the clusters.
Here you can add and revoke SSH keys. To add a ssh key, please follow these steps:

1. '''Select the cluster''' for which you want to create a second factor:</br> → [https://login.bwidm.de/user/ssh-keys.xhtml '''bwUniCluster 2.0''']</br> → [https://bwservices.uni-heidelberg.de/user/ssh-keys.xhtml '''bwForCluster MLS&WISO''']
[[File:BwIDM-twofa.png|center|600px|thumb|My SSH Pubkeys.]]

3. Click the '''Add SSH Key''' or '''SSH Key Hochladen''' button.
[[File:Bwunicluster 2.0 access ssh keys empty.png|center|400px|thumb|Add new SSH key.]]

4. A new window will appear.
Enter a name for the key and paste your SSH public key (file <code>~/.ssh/<filename>.pub</code>) into the box labelled "SSH Key:".
Click on the button labelled '''Add''' or '''Hinzufügen'''.
[[File:Ssh-key.png|center|600px|thumb|Add new SSH key.]]

5. If everything worked fine your new key will show up in the user interface:
[[File:Ssh-success.png|center|800px|thumb|New SSH key added.]]

Once you have added SSH keys to the system, you can bind them to one or more services to use either for interactive logins ('''Interactive key''') or for automatic logins ('''Command key''').

== Registering an Interactive Key ==

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
Interactive SSH Keys are not valid all the time, but only for one hour after the last 2-factor login.
They have to be "unlocked" by entering the OTP and service password.
|}

'''Interactive Keys''' can be used to log into a system for interactive use.
Perform the following steps to register an interactive key:

1. [[Registration/SSH#Adding_a_new_SSH_Key|'''Add a new interactive SSH key''']] if you have not already done so.

2. Select '''Registered services/Registrierte Dienste''' from the top menu and click '''Set SSH Key/SSH Key setzen''' for the cluster for which you want to use the SSH key.
[[File:BwIDM-registered.png|center|600px|thumb|Select Cluster for which you want to use the SSH key.]]

3. The upper block displays the SSH keys currently registered for the service.
The bottom block displays all the public SSH keys associated with your account.
Find the SSH key you want to use and click '''Add/Hinzufügen'''.
[[File:Ssh-service-int.png|center|800px|thumb|Add SSH key to service.]]

4. A new window appears.
Select '''Interactive''' as the usage type, enter an optional comment and click '''Add/Hinzufügen'''.
[[File:Ssh-int.png|center|600px|thumb|Add interactive SSH key to service.]]

5. Your SSH key is now registered for interactive use with this service.
[[File:Ssh-service.png|center|800px|thumb|SSH key is now registered for interactive use.]]

== Registering a Command Key ==

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
SSH command keys are always valid and do not need to be unlocked with a 2-factor login.
This makes these keys extremely valuable to a potential attacker and poses a security risk.
Therefore, additional restrictions apply to these keys:
* They must be limited to a single command to be executed.
* They must be limited to a single IP address (e.g., the workflow server) or a small number of IP addresses (e.g., the institution's subnet).
* They must be reviewed and approved by a cluster administrator before they can be used.
* Validity is reduced to one month.
|}

'''Command Keys''' can be used for automatic workflows.
Perform the following steps to register a "Command key":

1. [[Registration/SSH#Adding_a_new_SSH_Key|'''Add a new "command SSH key"''']] if you have not already done so.

2. Select '''Registered services/Registrierte Dienste''' from the top menu and click '''Set SSH Key/SSH Key setzen''' for the cluster for which you want to use the SSH key.
[[File:BwIDM-registered.png|center|600px|thumb|Select Cluster for which you want to use the SSH key.]]

3. The upper block displays the SSH keys currently registered for the service.
The bottom block displays all the public SSH keys associated with your account.
Find the SSH key you want to use and click '''Add/Hinzufügen'''.
[[File:Ssh-service-com.png|center|800px|thumb|Add SSH key to service.]]

4. A new window appears.
Select '''Command''' as the usage type.
Type the full command with the full path, including all parameters, in the '''Command''' text box.
Specify a network address, list, or range in the '''From''' text field (see [https://man.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man8/sshd.8#from=_pattern-list_ man 8 sshd] for more info).
Please also provide a comment to speed up the approval process.
Click '''Add/Hinzufügen'''.
{| class="wikitable"
! | Example
|-
| If you want to register a command key to be able to transfer data automatically, please use the following string as in the '''Command''' text field (please verify the path on the cluster first):
<pre>
/usr[/local]/bin/rrsync -ro / -rw /
</pre>
|}
[[File:Ssh-com.png|center|600px|thumb|Add interactive SSH key to service.]]

5. After the key has been added, it will be marked as '''Pending''':
You will receive an e-mail as soon as the key has been approved and can be used.
[[File:Ssh-service.png|center|800px|thumb|SSH key is now registered for interactive use.]]

== Revoke/Delete SSH Key ==

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
Revoked keys are locked and can no longer be used.
|}

'''SSH keys''' are generally managed via the '''My SSH Pubkeys''' menu entry on the registration pages for the clusters.
Here you can add and revoke SSH keys. To revoke/delete a ssh key, please follow these steps:

1. '''Select the cluster''' for which you want to create a second factor:</br> → [https://login.bwidm.de/user/ssh-keys.xhtml '''bwUniCluster 2.0''']</br> → [https://login.bwidm.de/user/ssh-keys.xhtml '''bwForCluster MLS&WISO''']
[[File:BwIDM-twofa.png|center|600px|thumb|My SSH Pubkeys.]]

2. Click '''REVOKE/ZURÜCKZIEHEN''' next to the SSH key you want to revoke.
[[File:Ssh-success.png|center|800px|thumb|Revoke SSH key.]]

File:BwServices my tokens.png

2021-06-19T11:11:24Z

S Siebler:

BwForCluster User Access

2021-06-09T13:45:59Z

S Siebler: update 2FA for MLS&WISO

Granting user access to a bwForCluster requires 3 steps:

# [[File:Zas assignment icon.svg|25px|]] Become part of an ''rechenvorhaben (RV)'' either by joining as a coworker or creating a new one
# [[File:bwfor entitlement icon.svg|25px|]] Permission from your university to use a bwForCluster called [[BwForCluster_Entitlement| bwForCluster entitlement]]
# [[#Personal registration at bwForCluster | Personal registration at the cluster site ]] based on approved ''RV'' [[File:Zas assignment icon.svg|25px|]] and issued bwForCluster entitlement [[File:bwfor entitlement icon.svg|25px|]].

{| style="width: 100%; border-spacing: 5px;"
| style="text-align:center; color:#000;vertical-align:middle;font-size:75%;" |[[File:Bwforreg.svg|center|border|500px|]]
|-
| style="text-align:center; color:#000;vertical-align:middle;" |
|}

<br>
Step 1 and 2 can be done at the same time. When both are finished, you can do step 3. Which cluster you will get access to depends on your research area and is decided in step 1.

= <b> ''RV'' registration at ''ZAS'' </b>=

Typically an RV is registered by the leader or a senior scientist of a research group/collaboration.

Any amount of co-workers can then join your RV to do work using the cluster.

== Register a new "RV" ==

In a RV, you will be asked to shortly describe your group's compute activities and the resources you need.

If you register your own RV, you will be:
# held accountable for the co-workers in the RV
# asked to provide information for the two reports required by the DFG for their funding of bwFor clusters
# likely asked for a contribution to a future DFG grant proposal for a new bwFor cluster in your area of research ("wissenschaftliches Beiblatt")

Please follow the steps at [[bwForCluster RV registration]]

== Become Coworker of an "RV"==

Your advisor (the "RV" responsible") will provide you with the following data on the RV:
* acronym
* password

To become coworker of an ''RV'', please login at
* https://www.bwhpc-c5.de/en/ZAS/bwforcluster_collaboration.php
and provide acronym and password. You will be assigned to the 'RV' as a member.

After submitting the request you will receive an email from ''ZAS'' about the further steps (i.e. [[#Personal registration at bwForCluster | personal registration at assigned bwForCluster]]).
The RV owner and any managers will be notified automatically.
You can see your RV memberships at https://www.bwhpc-c5.de/en/ZAS/info_rv.php
<br>

= <b> Permission of Your University ("bwForCluster entitlement") </b> =

Your own high school or university has to grant you permission to use a bwForCluster.

Getting permission from your university to calculate on bwForClusters is independent of an RV and can be done before or while getting an RV. If you are only creating an RV for your research group but do '''not''' plan to use the cluster yourself, you do not need to do this step.

Each university has their own procedure.

Please continue to the next step on this page and log in to a registration server. This will show you the list of your entitlements.
If the list contains <pre> http://bwidm.de/entitlement/bwForCluster </pre> you already have the entitlement.

The page [[bwForCluster Entitlement]] contains a list of participating universities and links to instructions on how to get an bwForCluster entitlement at each of them.

= <b> Personal registration at a bwForCluster - account creation</b> =

<b>Prerequisites for successful account creation:</b>
* [[BwForCluster_User_Access#RV_registration_at_ZAS|Membership in an RV (belonging to the bwForCluster you plan to join)]].
* [[BwForCluster_User_Access#Permission_of_Your_University_.28.22bwForCluster_entitlement.22.29|bwForCluster entitlement (assigned by your university)]].

Once you have registered your own RV (''rechenvorhaben'')
or a membership in an RV, you will receive
an email with a website to create an account for yourself
on that cluster. This email will send you to one of the following websites:

Available bwForCluster registration servers (service providers):
{| style="border:3px solid darkgray; margin: 1em auto 1em auto;" width="72%"
|-
!scope="row" {{Darkgray}} | Cluster topic and location
!scope="row" {{Darkgray}} | Registration server (for account creation)
|-
| bwForCluster JUSTUS 2 Ulm for Computational Chemistry and Quantum Sciences
| https://login.bwidm.de
|-
| bwForCluster MLS&WISO (Production and Development)
| https://bwservices.uni-heidelberg.de
|-
| bwForCluster NEMO Freiburg
| https://bwservices.uni-freiburg.de
|-
| bwForCluster BinAC Tübingen
| https://bwservices.uni-tuebingen.de
|}

In this chapter, a user account will be created at the cluster based on your personal credentials.

After having completed chapters 1 and 2 (RV approval and bwForCluster entitlement) please visit the

* bwForCluster ''service provider'' registration website (see table above or email after RV approval):
*# Select your home organization from the list of organizations and click '''Proceed'''.
*# You will be redirected to the ''Identity Provider'' of your home organization.
*# Enter your home-organizational user ID (might be user name, email, ...) and password and click '''Login''' or '''Anmelden'''.
*# When doing this for the first time you need to accept that your personal data is transferred to the ''service provider''.
*# You will be redirected back to the cluster registration website.
*# '''JUSTUS 2 only''': This step is required before registration for JUSTUS 2: To improve security a 2-factor authentication mechanism (2FA) is being enforced. You can manage your 2FA tokens by clicking on [https://login.bwidm.de/user/twofa.xhtml this link] or on '''My Tokens''' in the main menu of the JUSTUS 2 registration website. The instructions for registering a new 2FA token can be found on the following page: [[BwForCluster User Access/2FA Tokens]]. Please create at least one 2FA token before proceeding with JUSTUS 2 registration.
*# '''MLS&WISO only''': This step is required before registration for MLS&WISO: To improve security a 2-factor authentication mechanism (2FA) is being enforced. You can manage your 2FA tokens by clicking on [https://bwservices.uni-heidelberg.de/user/twofa.xhtml this link] or on '''My Tokens''' in the main menu of the bwServices registration website. The instructions for registering a new 2FA token can be found on the following page: [[BwForCluster MLS&WISO Production 2FA tokens]]. Please create at least one 2FA token before proceeding with MLS&WISO registration.
*# Select '''Service description''' within the box of your '''designated cluster'''. If the cluster is not visible, the reasons are either a missing entitlement or you are not member in an RV assigned to
that cluster - see prerequisites at the beginning of this chapter.
*# Click on the '''Register''' link below the service description to register for this cluster.
*# Make sure all requirements are met by checking the '''Requirements''' box at the top. If the requirements are not met you might be able to correct the issue by following the instructions. In all other cases please [http://www.support.bwhpc-c5.de open a ticket at the bwSupport Portal]. [[File:BwUniCluster 2.0 access login bwidm registration requirements.png|center|border|]]
*# Read and accept the terms and conditions of use ('''[v] I have read and accepted the terms of use''') and click on button '''Register'''. When requirements are missing, i.e. a missing second factor for authentication, you may need to correct that before being able to click on button "Register".
*# Click on '''Set Service Password''' and set a password for the cluster. '''Note: Setting a SERVICE password is MANDATORY for access to any bwForCluster. Using the password of your home organization is not accepted anymore.'''
*# Finally you will receive an email with instructions how to login to the cluster. Please wait at least 15 minutes before trying to login. More details about cluster login can be found in the next chapter. '''Note: Carefully read the email send by the registration server after account registration.'''
*# '''Note:''' You can return to the registration website at any time, in order to review your registration details, change/reset your service password or deregister from the service by yourself.

= <b> Login to bwForCluster </b> =

Personalized details about how to login to the cluster are included
in an email send after registration at the bwForCluster service provider.

General instructions for the bwForCluster login can be found here:
{| style="border:3px solid darkgray; margin: 1em auto 1em auto;" width="73%"
|-
!scope="row" {{Darkgray}} | Cluster topic and location
!scope="row" {{Darkgray}} | Login instructions
|-
| bwForCluster Chemistry JUSTUS 2 Ulm for Computational Chemistry and Quantum Sciences
| [[BwForCluster_JUSTUS2_Login|bwForCluster JUSTUS 2 Login]]
|-
| bwForCluster MLS&WISO Production
| [[BwForCluster_MLS&WISO_Production_Login|bwForCluster MLS&WISO Production Login]]
|-
| bwForCluster MLS&WISO Development
| [[BwForCluster_MLS&WISO_Development_Login|bwForCluster MLS&WISO Development Login]]
|-
| bwForCluster NEMO Freiburg
| [[bwForCluster NEMO Login]]
|-
| bwForCluster BinAC Tübingen
| [[bwForCluster BinAC Login]]

|}

<br>

= <b> Costs and Funding </b> =
The usage of bwForCluster is free of charge. bwForClusters are customized to the requirements of particular research areas.

bwForClusters are financed by the DFG (German Research Foundation) and by the Ministry of Science, Research and Arts of Baden-Württemberg based on scientifc grant proposal (compare proposals guidelines as per Art. 91b GG).

----
[[Category:Access|bwForCluster]][[Category:bwForCluster_Chemistry]][[Category:bwForCluster_MLS&WISO_Production]][[Category:bwForCluster_MLS&WISO_Development]]

File:Sds-hd storage service.png

2020-04-22T07:22:19Z

S Siebler: S Siebler uploaded a new version of File:Sds-hd storage service.png

SDS@hd Infrastructure Schema

File:Sds smb mac login.png

2020-04-20T14:51:25Z

S Siebler:

File:Sds smb mac mountpath.png

2020-04-20T14:50:37Z

S Siebler:

File:Sds bwservices servicepassword.png

2020-04-20T14:32:12Z

S Siebler:

File:Sds-hd hpc connection.png

2018-08-22T14:00:38Z

S Siebler: S Siebler uploaded a new version of File:Sds-hd hpc connection.png

File:Sds-hd storage service.png

2018-04-23T16:49:55Z

S Siebler: S Siebler uploaded a new version of "File:Sds-hd storage service.png"

SDS@hd Infrastructure Schema

Main Page

2017-10-23T12:34:41Z

S Siebler:

__NOTOC__

{| style="width: 100%; border:1px solid #e7aa01; background:#e7d9b4;border-spacing: 2px;"
| style="width:280px; text-align:center; white-space:nowrap; color:#000;" |
<div style="font-size:135%; border:none; margin:0; padding:.2em; color:#000;">Online</div>
<div style="font-size:135%; border:none; margin:0; padding:.2em; color:#000;">'''User and Best Practice Guides'''</div>
<div style="font-size:105%; border:none; margin:0; padding:.2em; color:#000;">of</div>
<div style="font-size:120%; border:none; margin:0; padding:.2em; color:#000;">Baden-Württemberg's HPC services</div>
|}



<br/>

{| style="width: 100%; margin:4px 0 0 0; background:none; border-spacing: 0px;"

| style="width:50%; border:1px solid #e7aa01; background:#f5fffa; vertical-align:top; color:#000;" |
{| style="width:100%; vertical-align:top; background:#f5fffa;"
| style="padding:2px;" | <div style="margin:3px; background:#cef2e0; font-size:120%; font-weight:bold; border:1px solid #e7aa01; text-align:left; color:#000; padding:0.2em 0.4em;">HPC Services</div>
|-
| style="color:#000;" | <div style="padding:2px 5px">The federated HPC competence centers of tier 3 provide and maintain
user guides and best practice guides for the compute clusters of '''[[:Category:bwHPC infrastructure|tier 3]]''':</div>

* <div style="font-size:115%;font-weight:bold;"> [[:Category:bwUniCluster|bwUniCluster]]
</div>
* <div style="font-size:115%;font-weight:bold;"> [[:Category:bwForCluster_Chemistry|bwForCluster JUSTUS]]
</div>
* <div style="font-size:115%;font-weight:bold;"> [[:Category:bwForCluster_MLS&WISO|bwForCluster MLS&WISO]]
</div>
* <div style="font-size:115%;font-weight:bold;"> [[:Category:bwForCluster NEMO|bwForCluster NEMO]]
</div>
* <div style="font-size:115%;font-weight:bold;"> [[:Category:bwForCluster BinAC|bwForCluster BinAC]]
</div>


<div style="padding:2px 5px">User and best practice guides for compute cluster of higher HPC tiers in Baden-Württemberg can be found here:
* bwHPC tier 1: [https://wickie.hlrs.de/platforms/index.php/Cray_XC40 Hazel Hen]
* bwHPC tier 2: [https://wiki.scc.kit.edu/hpc/index.php/Category:ForHLR ForHLR]

|}
| style="border:1px solid transparent;" |

| style="width:50%; border:1px solid #e7aa01; background:#f5faff; vertical-align:top;"|
{| style="width:100%; vertical-align:top; background:#f5faff;"
| style="padding:2px;" | <div style="margin:3px; background:#cedff2; font-size:120%; font-weight:bold; border:1px solid #e7aa01; text-align:left; color:#000; padding:0.2em 0.4em;">HPC Data Storage Services</div>
|-
| style="color:#000; padding:2px 5px;" | <div style="padding:2px 5px">
For user guides of the data storage services:</div>

* <div style="font-size:115%;font-weight:bold;">[[bwFileStorage]]
</div>
* <div style="font-size:115%;font-weight:bold;">[[sds-hd|SDS@hd]]
</div>
|-
|}
|}

<br>

File:Sds-hd-smb-auth.png

2017-09-20T09:14:56Z

S Siebler: SDS@hd SMB Authentication

SDS@hd SMB Authentication

File:Sds-hd-smb-netdrive.png

2017-09-20T09:14:12Z

S Siebler: SDS@hd Mapping SMB Networkdrive

SDS@hd Mapping SMB Networkdrive

File:Sds-hd hpc connection.png

2017-09-19T09:45:00Z

S Siebler:

File:Sds-hd storage service.png

2017-09-19T09:28:06Z

S Siebler: SDS@hd Infrastructure Schema

SDS@hd Infrastructure Schema