SDS@hd/Registration/Institutional Membership

2026-10-05T11:09:31Z

H Schumacher: H Schumacher moved page SDS@hd/Registration/Entitlement to SDS@hd/Registration/Institutional Membership without leaving a redirect

== Get SDS@hd Entitlement ==

New SVs can only be opened by bwIDM members who have the '''sds-hd-sv''' entitlement from their home university. This entitlement may have to be applied for individually at the home organisation. The criteria for awarding the entitlement to a user are entirely up to the respective home organization

To see if you already have the needed entitlement, you can [[SDS@hd/Registration/Entitlement#Check_your_Entitlements|''check your entitlements'']].

If you need the entitlement, follow the instructions for your home university. If none are provided, please contact your local service desk.

* '''Universität Freiburg'''
** Employees of the University of need to fill out this [https://uni-freiburg.de/forms-en/sds-hd-sv-entitlement/ '''form'''].
* '''Universität Heidelberg'''
** Employees of Heidelberg University receive this entitlement automatically. Students can't receive it.
* '''Universität Mannheim'''
** Employees of the University of Mannheim receive this entitlement automatically. Students can't receive it.
* '''Universität Ulm'''
** Please contact the helpdesk. See https://www.uni-ulm.de/einrichtungen/kiz/service-katalog/beratung-schulung-hilfe/helpdesk/

== Check your Entitlements ==

# Log in to https://bwservices.uni-heidelberg.de
# Go to "Index" -> "Personal Data"
# Choose the "Shibboleth" tab
# Search for the following entry: http://bwidm.de/entitlement/sds-hd-sv (this is an attribute and not a link)
If you can find it, you already have the needed entitlement.

→ [[SDS@hd/Registration#b)_Apply_for_new_SV| go forward with the SV application]]

SDS@hd/Access

2026-10-05T11:04:30Z

H Schumacher: changed prerequisites and added hint to them in troubleshooting section

This page provides an overview on how to access data served by SDS@hd. To get an introduction to data transfer in general, see [[Data_Transfer|data transfer]].

'''Notation:''' Please replace all placeholder contents within guillemets <code>< ></code> with the actual value. For example, replace ''<name_of_my_dog>'' with ''Fluffy'' .

== Prerequisites ==

* The two [[SDS@hd/Registration|registration]] steps are completed and still valid.
* Your IP address is within the [https://www.belwue.de BelWü] network which connects universities and other scientific institutions in Baden-Württemberg. If your computer is in your university network (e.g. at your office), you should be able to connect without restrictions. If it doesn't work or you are outside the BelWü network (e.g. at home), a VPN (virtual private network) connection to your university network must be established first. Please consult the VPN documentation of your university.
*: Exception: If you registered via eduGAIN or ORCiD, you don't need to be part of a specific network.
* If you encounter any issues, please have a look at the [[#Troubleshooting|Troubleshooting]] section.

== Needed Information, independent of the chosen tool ==

* '''Username:''' You can look it up under your user data at [https://bwservices.uni-heidelberg.de/user/index.xhtml bwServices]. The username is ''<localPrimaryGroup>_<eduPersonPrincipalName>'' . (It is the same as for the bwHPC Clusters. Example: ''hd_ab123''.)
* '''Password:''' The Service Password that you set at bwServices in the [[SDS@hd/Registration#Step_B:_Registration_for_SDS@hd_Service|registration step]].
* '''SV-Acronym:''' Use the lower case version of the acronym for all access options.
* '''Hostname:''' The hostname depends on the chosen network protocol:
** For [[Data_Transfer/SSHFS|SSHFS]] and [[Data_Transfer/SFTP|SFTP]]: ''lsdf02-sshfs.urz.uni-heidelberg.de''
** For [[SDS@hd/Access/SMB|SMB]] and [[SDS@hd/Access/NFS|NFS]]: ''lsdf02.urz.uni-heidelberg.de''
** For [[Data_Transfer/WebDAV|WebDAV]] the url is: ''https://lsdf02-webdav.urz.uni-heidelberg.de''
* '''Domain:''' The domain is ''BWSERVICESAD'' (needed for smb and nfs connections).

== Recommended Setup ==
The following graphic shows the recommended way for accessing SDS@hd via Windows/Mac/Linux. The table provides an overview of the most important access options and links to the related pages. 
If you have various use cases, it is recommended to use [[Data_Transfer/Rclone|Rclone]]. You can copy, sync and mount with it. Thanks to its multithreading and checksumming capability Rclone is a good fit for transferring big data. 
For an overview of all connection possibilities, please have a look at [[Data_Transfer/All_Data_Transfer_Routes|all data transfer routes]].

[[File:Data_transfer_diagram_simple.jpg|center|700px]]
Figure 1: SDS@hd main transfer routes
*Institutions that allow SMB connections to SDS@hd are for example: Heidelberg University, Karlsruhe Institute of Technology (KIT), Hohenheim University, Freiburg University, Hochschule Rottenburg, Hochschule Biberach. Currently, SMB connections are not possible from Mannheim University.

{| class="wikitable"
|- style="font-weight:bold; text-align:center; vertical-align:middle;"
!
! Use Case
! Windows
! Mac
! Linux
! Possible Bandwith
! Firewall Ports
|-
| [[Data_Transfer/Rclone|Rclone]] + <protocol>
| copy, sync and mount, multithreading, checksumming for verifying success of up-/download
| ✓
| ✓
| ✓
| depends on used protocol
| depends on used protocol
|-
| [[SDS@hd/Access/SMB|SMB]]
| mount as network drive in file explorer or usage via Rclone
| [[SDS@hd/Access/SMB#Windows|✓]]
| [[SDS@hd/Access/SMB#Mac|✓]]
| [[SDS@hd/Access/SMB#Linux|✓]]
| up to 40 Gbit/sec
| 139 (netbios), 135 (rpc), 445 (smb)
|-
| [[Data_Transfer/WebDAV|WebDAV]]
| go to solution for restricted networks
| [✓]
| ✓
| ✓
| up to 100GBit/sec
| 80 (http), 443 (https)
|- style="vertical-align:middle;"
| [[Data_Transfer/Graphical_Clients#MobaXterm|MobaXterm]]
| Graphical User Interface (GUI)
| [[Data_Transfer/Graphical_Clients#MobaXterm|✓]]
| ☓
| ☓
| see sftp
| see sftp
|- style="vertical-align:middle;"
| [[SDS@hd/Access/NFS|NFS]]
| mount for multi-user environments
| ☓
| ☓
| [[SDS@hd/Access/NFS|✓]]
| up to 40 Gbit/sec
| -
|- style="vertical-align:middle;"
| [[Data_Transfer/SSHFS|SSHFS]]
| mount, needs stable internet connection
| ☓
| [[Data_Transfer/SSHFS#MacOS_&_Linux|✓]]
| [[Data_Transfer/SSHFS#MacOS_&_Linux|✓]]
| see sftp
| see sftp
|- style="vertical-align:middle;"
| [[Data_Transfer/SFTP|SFTP]]
| interactive shell, better usability when used together with Rclone
| [[Data_Transfer/SFTP#Windows|✓]]
| [[Data_Transfer/SFTP#MacOS_&_Linux|✓]]
| [[Data_Transfer/SFTP#MacOS_&_Linux|✓]]
| up to 40 Gbit/sec
| 22 (ssh)
|}
Table 1: SDS@hd transfer routes

=== Access from a bwHPC Cluster ===

'''bwUniCluster''' 
You can't mount to your $HOME directory but you can create a mount under $TMPDIR by following the instructions for [[Data_Transfer/Rclone#Usage_Rclone_Mount | Rclone mount]]. It is advised to wait a couple of seconds (<code>sleep 5</code>) before trying to use the mounted directory.

'''bwForCluster Helix''' 
You can directly access your storage space under ''/mnt/sds-hd/'' on all login and compute nodes.

'''bwForCluster BinAC 2''' 
You can directly access your storage space under ''/mnt/sds-hd/'' on all login and compute nodes. The prerequisites are:
* The SV responsible has enabled the SV on BinAC 2 once by writing to [mailto:sds-hd-support@urz.uni-heidelberg.de sds-hd-support@urz.uni-heidelberg.de]
* You have a valid kerberos ticket, which can be fetched with <code>kinit <userID></code>

'''Other''' 
You can mount your SDS@hd SV on the cluster yourself by using [[Data_Transfer/Rclone | Rclone]] with the [[Data_Transfer/Rclone#Usage_Rclone_Mount | Rclone mount]] command. As transfer protocol you can use WebDAV or sftp. For a full overview please have a look at [[Data_Transfer/All_Data_Transfer_Routes | All Data Transfer Routes]].

=== Access via Webbrowser (read-only) ===

Visit [https://lsdf02-webdav.urz.uni-heidelberg.de/ lsdf02-webdav.urz.uni-heidelberg.de] and login with your SDS@hd username and service password. Here you can get an overview of the data in your "Speichervorhaben" and download single files. To be able to do more, like moving data, uploading new files, or downloading complete folders, a suitable client is needed as described above.

== Best Practices ==

* '''Managing access rights'''
** When the SV is mounted on Windows, access rights can be adjusted in the file/folder properties menu. (Users from Heidelberg without Windows can use the [https://www.urz.uni-heidelberg.de/de/service-katalog/desktop-und-arbeitsplatz/windows-terminalserver Windows terminal server]).
** bwForCluster Helix users can change the access rights via Helix by using [[Workspace#Regular_Unix_Permissions | unix permissions]] or [[Workspace#ACLs:_Access_Control_Lists | access control lists]]. On non native mounts ACL changes won't work.
** Additionally to the already mentioned options, the SVV can take over ownership of files by opening a [[Data_Transfer/SFTP|SFTP]] shell and running <code>chown -R <userID> <path/to/folder></code> .

* '''Multiuser environment''' → Use [[SDS@hd/Access/NFS|NFS]]

== Troubleshooting ==

Maintance windows and known issues are communicated via the email list or the News section at the SDS@hd start page in the wiki.

* '''Issue:''' The credentials aren't accepted
*: Check if your credentials work in general by trying the [https://lsdf02-webdav.urz.uni-heidelberg.de/ access via webbrowser].
*: If this doesn't work:
*:: → Double check the [[#Prerequisites | Prerequisites]]
*:: → [[Registration/Login/Username| Check your Username]]
*:: → [[Registration/bwForCluster/Helix#Troubleshooting_with_the_Help_of_bwServices | Troubleshooting with bwServices]]
* '''Other Issue'''
** If available, follow the troubleshooting guide of your specific connection method.
** Make sure to not use LAN and WLAN at the same time to prevent connection problems.
** If you have an institutional account, make sure to be a fully active member of your institution.
If these suggestions didn't help, write to the [mailto:sds-hd-support@urz.uni-heidelberg.de support]. Provide the following information:
* Your operating system
* Does the access via webbrowser work?
*: If yes, provide us with detailed information on how you tried to access your SV (used access method, username, ...).

SDS@hd/Registration

2026-09-28T14:14:05Z

H Schumacher: added link to instructions for PI/PM

== Prerequisites ==

* Only scientific data is stored. Don't place administrative data on SDS@hd.
* Only hot data is stored which means only data that is actively used is allowed. We expect you to remove data from finished projects. It is advised to use an archiving service for this. Temporary data should also be removed on a regular basis.
* The storage of personal data or data capable of being linked to an individual is not permitted.
* SDS@hd mustn't be used as an archive or a backup. To find solutions in this regard, please have a look at the [https://wiki.bwhpc.de/e/Main_Page Main Page] and your institutional services.
* The use of SDS@hd must be adequately documented:
** All publications that were created with the involvement of the storage service SDS@hd should acknowledge its use. For an example text see [[SDS@hd/Acknowledgement | here]].
** These publications, scientific posters, theses and third-party funded projects for which the service was used need to be reported to us via [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] or via [mailto:sds-hd-support@urz.uni-heidelberg.de email].

== Registration Steps ==

The registration consists of two steps:

* '''Step A:''' Membership in Storage Project (Speichervorhaben/'''SV''', same as SDS@hd allocation). There are two options:
*: '''a)''' →[[#a)_Get_invited_to_existing_SV | Get invited to existing SV]]
*: '''b)''' →[[#b)_Apply_for_new_SV | Apply for new SV]]
* '''Step B:''' →[[#Step_B:_Registration_for_SDS@hd_Service |Registration for SDS@hd Service]]

After finishing the registration, the '''next steps''' are:
* Check which management options there are for your respective role in the SV → [[SDS@hd/SV_Management |SV Management]]
* See how to access and use your storage space → [[SDS@hd/Access | Access]]

[[File:Sds-hd-registration_new.png|center|x500px|upright|SDS@hd Registration Process]]

=== Step A: SV Membership ===
[[File:BwServices_Management_project_sv_structure.png|600px|thumb|right|A Project consists of multiple elements like a description, users and allocations/SVs. Every user in the project can be added to one or multiple of the available SVs.]]
You can either join an existing SV or start a new one. SVs are always part of a Project at the [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] page. In the image on the right you can see how the general setup of a Project looks like. The person who created the Project is called PI (Principal Investigator). The PI and the PMs (Project Managers), defined by the PI, can invite new users to the project and add them to SVs.

==== a) Get invited to existing SV ====

To join an existing SV as a coworker, ask the PI or a PM to...:
# ...add you to the project:
#* They can add you directly if you've already logged in to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] once. You get notified via email.
#* They can invite you by sending an email invite. You have to accept the invite to be a project member first, before you can be added to the SV.
# ...add you to the SV.
#*: You can see that you were added by going to the page of the bwServices-Project on [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] and then checking the list of allocations that you are a member of. You get notified via email as well.

Hint: Instructions for the PI/PM can be found under [[SDS@hd/SV_Management#Adding_users | Adding Users]]

→ As soon as you're a member of an SV, you can [[#Step_B:_Registration_for_SDS@hd_Service | go on with Step B]] if you are using SDS@hd for the first time.
If not, it's a good idea to trigger an update of your user data by logging in to bwServices once.

==== b) Apply for new SV ====

This is typically done only by the leader of a scientific work group or the senior scientist of a research group/collaboration.
You can add any amount of co-workers to your SV.

# '''Check prerequisites:''' Your institution must be a [https://www.bwidm.de/hochschulen.php bwIDM member] and you have to be an employee of your institution.
# '''Create a Project''' at [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] (alternatively use an already existing project):
## Go to the bwServices-Management Tool. At the start page click the title "Projects" and then "Add a project".
## Fill in and submit the form:
##* Keep in mind that this bwServices-Project can stand for a scientific research project but alternatively, it can for example represent all acitivities of a research group as well. The title and description can be changed later.
##* The information to fill in is as follows: <syntaxhighlight>
Please provide a detailed description of your work and select your field of science. In case that this Project corresponds to one dedicated scientific project, please describe your scientific project and indicate the expected project duration. Please mention additional infrastructure that is used by the project (cluster, cloud, storage, archiving services). Thank you!
</syntaxhighlight>
##* The provided address should be the one where a potential bill can be sent to. In cases where different SVs in the project would have different billing addresses, the address field in the SV application can be filled in additionally.
#: → Your project is directly active.
# '''Apply for SV'''. This is the same as adding allocations to a project:
## In the Allocations section of the Project's page is a button "Request Resource Allocation". Currently, the only allocation that can be chosen is an allocation for the resource SDS@hd. An SDS@hd allocation is called SV (Speichervorhaben / storage project).
## Fill in the form:
##* Justification questions (this field is the only one that can't be changed later on): <syntaxhighlight>
Q1: If not already provided in the bwServices-Management-Project description: Describe the intended usage of the storage space and indicate the expected duration that the allocation is needed.
Q2: How are data generated and processed? Please distinguish primary vs. secondary data and mention relevant equipment or software.
Q3: What is the expected data growth? Which file formats are used? Please describe any notable characteristics of your data, such as exceptionally large file size or very large number of small files.
Q4: Describe at which timepoints data can be archived or deleted. Describe the data and the data volume that can be archived at the end of the project.
Q5: How are your project data structured, and how is access managed? Are there special needs for the access rights within the SV (e.g. users or guest folder)?
</syntaxhighlight>
##* Title of the SV: The title is for you to recognize the SV in the allocations table if you have multiple SVs. It can be short.
## After filling in the form and agreeing to the terms, the allocation is shown on the project page with the status 'New'. A 'New' allocation is not approved, yet, and therefore not usable.
## The allocation application gets reviewed and approved. In the approval state no new users can be added. As soon as the storage space was set up, an email informs about the activation of the allocation.

→ [[#Step_B:_Registration_for_SDS@hd_Service | Go on with Step B]] if you are using SDS@hd for the first time.

If you're a PI in a project with SDS@hd allocations, you will be:
* ...held accountable for the co-workers in the SV
* ...responsible for all data stored in this SV, as well as for any costs incurred as a result.
* ...asked to provide information for the two reports required by the DFG for their funding of SDS@hd
* ...likely asked for a contribution to a future DFG grant proposal for an extension of the storage system in your area of research ("wissenschaftliches Beiblatt")

=== Step B: Registration for SDS@hd Service ===
[[File:bwServices_chooseHomeOrganisation.png|right|300px|thumb|Select your home organization]]

After step A you have to register your personal account on the storage system and set a service password.

* Visit the registration website '''bwServices''': [https://bwservices.uni-heidelberg.de/ https://bwservices.uni-heidelberg.de]
*# Select your home organization from the list and click ''Proceed''
*#: You will be directed to the ''Identity Provider'' of your home organisation
*# Enter your home-organisational user ID / username and your home-organisational password and click ''Login'' button
*#: You will be redirected back to bwServices
* '''Register''' for the SDS@hd service:
*# Under ''The following services are available'' select the service ''SDS@hd - Scientific Data Storage''.
*# Click ''Register''
* Set your personal '''service password''' for SDS@hd. This is used for accessing SVs.
* The registration is finished.
*: → [[#sds_registration_nextSteps|next steps]]

=== Change Service Password at bwServices ===

→ '''[[Registration/bwForCluster/Helix#Setting_a_New_Service_Password | Setting a New Service Password]]'''

== FAQ ==
* Is there a Test-SV to do technical tests?
*: Yes, there is a Test-SV for this purpose. You can write to the support to get invited. Please keep in mind that data gets regularly deleted and users may be removed after some time.
* I can login with different institutions/IDs (edugain, ORCID, university ID, university clinics ID,...). Which one shall I choose?
*: If you want to be able to start your own SVs, use the ID were you can get the entitlement most conveniently. Otherwise it doesn't matter. But ORCID is the most unspecific option and therefore should only be chosen if the other options don't work.
* I own a SV with my ID at institution A. Can I connect my account from institution B to it?
*: You can't connect different accounts. But you can join the SV with both accounts if necessary.
* What is my userID when I register via ORCID?
*: You'll be informed about the ID along the process. It'll start with "orcid_".
* How do I set up ORCID?
*: Login with your ORCID account (instead of an institutional account) and set it up so that your email address is shared with us.
* How do I register with foreign institutions (outside of [https://www.belwue.de/ueberuns/teilnehmer.html BelWü])
*: We might need to enable the IP range first.
*: After we've added the ip addresses, you might need to wait for an hour before it works.

== Troubleshooting ==

* '''Issue:''' The registration on bwServices is not possible
** Error message: You have no access to a Speichervorhaben of the Service SDS@hd. Please register a new Speichervorhaben or collaborate with an existing Project on https://management.bwservices.uni-heidelberg.de/
** Solution: Check the following points:
*** Are you a member of a bwServices-Project?
*** Does this project have an SDS@hd allocation?
*** Are you a member of this allocation/SV?
*** Is the bwServices-Project still active? The project mustn't be in the review process.
*** Is the allocation still active? The allocation mustn't be expired.

SDS@hd/SV Management

2026-09-28T14:11:18Z

H Schumacher: moved section for adding users, added instructions

{| style=" background:#FEF4AB; width:100%;"
| style="padding:8px; background:#FFE856; font-size:120%; text-align:left" | For (previous) users of the SDS@hd Management Tool:
|-
|
On the 1st of September we're going to replace the SDS@hd Management website with the [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] website. 
The '''main changes in the registration and management process''' are as follows:
* An SV must be a part of a bwServices-Project.
* An SV is an ''Allocation'' requested from the ''Resource'' SDS@hd.
* The SVV role was removed. SVVs are now the PI (Principal Investigator) of a bwServices-Project that contains the SV.
* The SV-Manager role was removed. Managers (PMs) can be assigned within the project and have mostly the same rights as the PI.
* Publications can be added to the project at any time.
* New SV members can directly be added by one of the PMs. There is no need for sharing an SV-password and filling in a collaboration form anymore.
* New SV members are always added as 'Users' (= full members). If you want to add guests, please directly change their role to 'Guest' after adding them!

'''Important points for the migration:'''
The data of current SVs were migrated to the new page:
* The SVV is now the PI of a project and the project has an SDS@hd allocation.
* Inactive members of the SVs were removed.
* <ins>Initial tasks for the PIs</ins>:
*: Login to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] and update the following points:
** In the project:
*** Check and update the data (project description and attributes).
*** Define managers. Read their role description first! Check their rights in the [[#Project_Manager_(PM) | context of the project]] and in [[#PMs | context of the allocation]] (SV).
** In the allocation/SV:
*** Check the members list and if applicable the guest roles.
*** Check the attributes of the allocation (access options, quota, ...). They can be changed via [[#Adjust_SV_Attributes | Change Request]].
*** Billing data must only be provided if the SV generates costs according to the [https://sds-hd.urz.uni-heidelberg.de/management/shib/en/sds_costs_download.php price sheet] and if additionally, the billing information is different from the information provided in the project.
* The SDS@hd Management page will stay in read-only mode for a short while before it gets deactivated.
* Test-SV: Members that weren't active in the last 30 days were removed. Access to the Test-SV is given on request to the SDS@hd Support.
* Inactive PI: We'll get in contact with projects that don't have an active PI.
|}
This page provides an overview of options to structure projects that use SDS@hd. The simple use case is:
* One research project has one bwServices-Management Project (in short bwServices-Project).
* One bwServices-Project has one SV.
* Both have the same group of members.
The graphic below visualizes one option for another setup. Depending on your preferences of who should have access to which data, you can structure things differently.
[[File:BwServices_Management_project_sv_structure.png|600px|thumb|right|A Project consists of multiple elements like a description, users and allocations/SVs. Every user in the project can be added to one or multiple of the available SVs.]]

= bwServices-Management =

In bwServices-Management the top level entity is a project. On this page ''project'' always refers to this ''bwServices-Project''. Every bwIDM member with employee status can create a project by going to the ''Projects'' page (Home -> Projects) and then clicking on the "Add a project" button. A project can have multiple users. For specifics regarding the different roles in a project see the [[#Roles_in_a_Project| Roles]] section.
[[File:bwServices_Management_home.png|300px|thumb|right|The start page lists recent projects and allocations. The titles link to the complete lists.]]

Here is an overview of the most important pages and what you can find there:
[[File:bwServices_Management_pagetree.png|600px|center|bwServices-Management Page Overview]]

== Roles in a Project ==
There are three different roles in a project. Depending on the role, different management activites can be executed at [https://management.bwservices.uni-heidelberg.de/ bwServices-Management]. These are described below. They include top level management of user access.

=== User ===
Users can...
* ...view project information.
* ...view allocations that they are a member of.

* ...enable/disable notifications. Notifications regarding the Projects that they are (or will be) a member of can be:
** information that they were added to the project
** emails from one of the project's PMs
** information about a project being archived

=== Project Manager (PM) ===
Additionally to the abilities of a User, a PM can...
* ...add and remove Users from the project except themselves or the PI.
* ...add and remove Users from allocations except themselves or the PI.
* ...change the roles of Users on project and allocation level (Project: User <-> PM; SV: None <-> Guest).

* ...enable/disable notifications for other Users.
* ...always receive email notifications.
* ...write an email to all Project Users.

* ...request new allocations.
* ...change project attributes.
* ...add publications.
* ...extend project lifetime.
* ...extend allocation lifetime.
* ...can archive (~ delete) the project.

==== Adding users ====
PMs can add users to their projects and to any allocation within the project. To provide a user with access to an allocation, both of the following steps must be followed.

'''1.''' Adding users to a project:
# On the project's page either the button on the top or next to the users list can be used for adding users.
# There are two ways to add new project members:
#:* Option 1: To invite someone that already logged in to the bwServices-Management page once, they can be found...
#:** ... via their user id. A list of ids can be provided in the search field.
#:** ... by providing (a part of) the name and searching "All Fields".
#:* Option 2: Alternatively, an invite link can be sent by providing the email address. The email address for sending the invite doesn't need to be the address that is later on used for their account on bwServices-Management. When a user clicks on the invite link they have to login to bwServices-Management. Afterwards, they are a part of the project with the account that they logged in with.
# Before adding the person or submitting the invite, the role of the new project member should be adjusted if needed. Available roles are "[[#User |User]]" or "[[#Project_Manager_(PM) |Manager]]".
# The added person gets an email notification regarding their new project membership.

A project can have multiple allocations. Having an allocation means that you can allocate a resource like SDS@hd for your project. An allocation comes with its own form to fill in and its own terms to agree to. You can find detailed instructions under [[SDS@hd/Registration#b)_Apply_for_new_SV | Registration/Apply for new SV]].

'''2.''' Adding users to an allocation:

An allocation can have multiple users. To add a new user:
# Go to the page of the allocation
# Go to the section "Users in Allocation" and click "Add Users"
# Choose the users that shall be added from the list of project members.
#: If you don't see the user here, please make sure that you've added them to the project first.

==== Yearly Project Extension ====

Once a year a PM in the project needs to apply for a lifetime extension. This is indicated by the label "Project Review". Please update your project information first before applying for the extension. Especially, update the list of publications for which the allocated resources (e.g. SDS@hd) were used.

==== Archiving a Project ====
Caution: Same as '''deleting''' all allocations!

For PMs there is a button for archiving the project in the upper right part of the project page. Archiving means that the status of the allocations changes to "expired" and the [[#SV_Deprovisioning | deprovisioning]] process starts. The project with its description will be deprovisioned some time after all allocations are removed.

=== Principal Investigator (PI) ===
A PI is automatically a PM as well. -> Mentions of 'PM' always include the PI.
Additionally to the abilities of a PM, the PI...
* ...is responsible for the whole project. This includes all allocations, independently of who requested them.
* ...is automatically a member of every allocation.

== Roles in a SV ==

The SV users keep their project roles (PI/PM/User). Additionally, they can hold the Guest role in an SV. The roles within the context of an SV are described below.

=== All SV Users ===
At [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] they can...
* ...view their role in the SV. If no Role column is shown, there are no users with a Guest role in the SV.
* ...view the status of the SV (active/expired, end date)
Notifications regarding the SVs that they are a member of can be:
* information about having been added to an allocation
* information about accepted or denied change requests (without the concrete content of the request)
* information about an allocation being activated or denied

=== Guests ===
PMs can add the "Guest" role to a user in the SV by going to the users list of the allocation and clicking the pencil button in the last column of the user's row. This leads to the details of the user where the role can be changed.
As soon as the first user is defined as Guest via bwServices-Management, a "guest" folder is created in the SV. A Guest can see but not access content in the SV root. They can only work within the "guest" folder. For more information see [[#Guest_Folder | Guest Folder]].
'''Caution:''' When adding new guests to the Project and afterwards to the SV, make sure to directly change their role in the SV from the default to "Guest". Otherwise they have full access first.

=== PMs ===
PMs in the project can...
* ...manage the SV as mentioned in their role description.
* ...adjust SV attributes via change request. Previous change requests can be viewed.
* ...extend the SV lifetime.
* ...request the deletion of the SV.

==== Adjust SV Attributes ====

To change the properties of your SV, you can hand in a Change Request.
When you request a quota expansion for your SV at the SDS@hd service in order to store hot scientific data, we kindly ask you to answer the questions in the justification textfield for reasoning the extension. The questions are:
<syntaxhighlight lang="text">Q1: Please explain why additional storage is needed.
Q2: What is the potential for deleting or archiving data?</syntaxhighlight>

Please also consider the following remarks:
Upon increasing storage capacities, there will also be a need for further activities in the reporting from your side, e.g. by pointing out the usage of SDS@hd in scientific publications to us. More information about this can be found here: https://wiki.bwhpc.de/e/SDS@hd/Acknowledgement
Please consider raising funds for the usage and further operation of the SDS@hd service. We will gladly support you with this.

==== Yearly SV Extension ====

Once a year a PM needs to apply for a lifetime extension of the storage project. This is indicated by the label "Allocation Renewal". As long as SDS@hd is used appropriately, an extension will be granted.
Warning: You have to apply for the extension in the given timeframe. When you're too late, you have to contact the sds-support to get the extension. SVs that don't get extended, are going to be deleted by us, see [[#SV_Deprovisioning | SV Deprovisioning]]. Before the deletion, the PI is informed via email.

==== SV Deprovisioning ====
The deprovisioning is initiated in the following cases:
* The SV is not extended.
* The Project was archived.
* A PM that is a member of the SV asks via Change Request for a deletion of the SV by writing this request in the "Justification for Changes" field.

The deprovisioning basically contains the following steps:
* The access of the SV-group to the SV is blocked. The access to any data that is saved within the SV is not possible anymore.
* After a specific period of time all data in the SV gets permanently deleted.

== FAQ ==

* I changed a user's role in bwServices-Management but the user still has the previous access rights.
*: After 30 minutes the adjustment should be synced to all systems. If not, it helps when the member with the role change logs in to bwServices once to trigger an automatic update and then waits for 30 minutes.
* How can I remove myself from an allocation or project?
*: This is not possible. You must ask a PM to remove you.
* How can I deactivate Users?
*: Users can't be deactivated. They can only be removed from allocations or projects.
* My project or my allocation ist expired. How can I prolong it?
*: Please [mailto:sds-hd-support@urz.uni-heidelberg.de submit a Ticket]
* How can the PI of a project be changed?
*: This is currently not possible but will be possible in the future.

= Within the SV =
While bwServices-Management manages the access rights and storage amount on a top level outside of the storage space itself, there are multiple options to organize data and adjust access rights when being connected to the SV.

== Default Access Rights ==

* In the main space of an SV everyone has read and write access.
* The person who creates a file or folder is the owner. The owner can adjust the rights via [[Workspace#Regular_Unix_Permissions | chmod]] when using one of the following connection methods: sftp/sshfs, nfs, bwVisu, bwForCluster Helix. On Windows the rights can be set with a right click on the file/folder and using the properties menu.
* The SV responsible has full control over the access rights by being able to take [[#Taking_Ownership_of_Files | ownership of files]].
* The SDS@hd support can help with defining access rules for your specific use case. Optimally, this is discussed before the SV gets filled with data. Examples:
** A group folder with read only access in contrast to the default read and write access. The SV members can see everything from each other but not change files that aren't their own.
** A users folder where every user can create their own folder with content that is inaccessible to others.

As an example, let's assume that the SV is structured as shown in the graphic below.
[[File:SDS_AccessRules_Overview.jpg|600px|center|Default access rules in a SV]]

In the SV scheme on the left you can see that the SV has two guests, two full members and the PI. The full members have read and write access to all normal folders in the SV root and the `guest` folder.

Additionally, there can be a `users` folder with a special setup. In the `users` folder everyone has read access but only the owners of the folders/files have write access.
<ins>Hint:</ins> If needed, the SDS@hd support can create custom access rules for specific top level folders.

The SV scheme on the right visualizes how the orange full member experiences the access when working in the SV. The full member has free read and write access to everything except the personal folder of the blue user.

== Guest Folder ==
Using the option to add some SV members with the role "Guest" allows better access management for separate user groups that shall not get access to the whole SV.

'''Use Cases:'''
* Data exchange with core facilities.
* Projects together with external collaborators.
* Up- and Download area

'''Setup:'''
* PMs can enable the guest folder by adding guests to the SV see [[#Guests | Guests]]. The folder gets created as soon as the first Guest user is defined. The creation might need up to 20 minutes.

'''Access Rights'''
* Full SV member:
** Read and write access to all content of the SV including the guest folder.
** No access to content that was deliberately made unaccessible by other SV members via <code>chmod</code>.
** No access to 'user folders'.
* Guest:
** Folder above guest folder:
*** Can see all file and folder names. Can't see content in subfolders.
*** Can't access or create content.
** Guest folder:
*** Can only create folders, no files, in the root of the guest folder.
*** Read content. Read content in subfolders.
**** Alternative setup: Read content. Can't read content in subfolders / folders from other guests. Please contact the sds-support to get this setup.
*** Has only write access to content that they created themselves.

The following graphic shows the current default access rights in the guest folder on the left in contrast to the alternative setup on the right.
[[File:SDS_AccessRules_Guest.jpg|600px|center|Guest folder default vs. custom]]

== Storage Space Management ==

At the moment you can't see how much storage space is left. We are working on including this information.

To keep the data hot, you should regularly check the following points:
* Make sure that temporary data gets removed.
* Make sure that no data from finished projects is stored. You might want to archive those with one of the archiving services listed on the [https://wiki.bwhpc.de/e/Main_Page Main Page].
* If someone has left the storage project, make sure that the PI has access to the data and the data is still hot.

Helix/Login

2026-09-28T09:45:10Z

H Schumacher: Added hint for SDS@ħd+bwVisu users.

= Prerequisites =
* The three [[registration/bwForCluster|bwForCluster registration]] steps are completed and still valid.
* Your IP address is within the [https://www.belwue.de BelWü] network which connects universities and other scientific institutions in Baden-Württemberg. If your computer is in your university network (e.g. at your office), you should be able to connect to bwForCluster Helix without restrictions. If you are outside the BelWü network (e.g. at home), a VPN (virtual private network) connection to your University network must be established first. Please consult the VPN documentation of your university.
* As credentials you need your Helix [[Helix/Login#Username|username]], [[Registration/Password|service password]] and [[Registration/2FA|2FA token]].
* If you encounter any issues, please have a look at the [[Helix/Login#Troubleshooting|Troubleshooting]] section.

= Login to bwForCluster Helix =

Login to bwForCluster Helix is only possible with a Secure Shell (SSH) client for which you must know your bwForCluster Helix credentials and the hostname of the login nodes.
For more general information on SSH clients, visit the [[Registration/Login/Client|SSH clients Guide]].

== Username ==

Your username on bwForCluster Helix consists of a prefix and your local username.
For prefixes please refer to the [[Registration/Login/Username|Username Guide]].

Example: If your local username at your University is <code>ab123</code> and you are a user from Heidelberg University, your username on the cluster is: <code>hd_ab123</code>.

== Hostnames ==

bwForCluster Helix has two login nodes. In general, you should stay with automatic node selection, so that the connections are evenly distributed between the login nodes. To do this, we use so-called DNS round-robin scheduling which takes care of the node balancing. This means, if you open multiple SSH sessions, these sessions may run on different login nodes. Therefore, processes started in one session might not be visible in other sessions.

{| class="wikitable"
! Hostname !! Destination
|-
| '''helix.bwservices.uni-heidelberg.de''' || one of the two login nodes
|}

== Login with SSH command (Linux, Mac, Windows) ==

Most Unix and Unix-like operating systems such as Linux and Mac OS as well as newer versions of MS Windows (10 and 11), have a built-in SSH client provided by the OpenSSH project.
If you want to use the full set of Linux commands on Windows (10 or 11), you can also install the [https://docs.microsoft.com/en-us/windows/wsl/install Windows Subsystem for Linux] (WSL).

For login use one of the following ssh commands in a commandline (i.e. cmd, terminal, PowerShell):

ssh <username>@helix.bwservices.uni-heidelberg.de
ssh -l <username> helix.bwservices.uni-heidelberg.de

To run graphical applications on the cluster, you need to enable X11 forwarding with the <code>-X</code> flag:

ssh -X -l <username> helix.bwservices.uni-heidelberg.de

To make the login process faster, tell the command that no ssh key authentication is used by adding an additional parameter:

ssh -o PubkeyAuthentication=no <username>@helix.bwservices.uni-heidelberg.de

== Login with graphical SSH client (Windows) ==

For Windows we suggest using [[Data_Transfer/Graphical_Clients#MobaXterm|MobaXterm]] for login and file transfer.

Start MobaXterm and fill in the following fields:
<pre>
Remote name : helix.bwservices.uni-heidelberg.de
Specify user name : <username>
Port : 22
</pre>

After that click on 'ok'. Then a terminal will open where you can enter your credentials.

== Login with SSH key ==

To set up a SSH key for your workflow, read [[Registration/SSH|Registering SSH Keys with your Cluster]].
To login by using the ssh key, use the following command:

ssh -o IdentitiesOnly=yes -i <path_to_private_key> <username>@helix.bwservices.uni-heidelberg.de

== Login Example ==

To login to bwForCluster Helix, proceed as follows:
# Login with SSH command or MoabXterm as shown above.
# The system will ask for a one-time password <code>Your OTP</code> (= Smartphone Token / TAN list element/ ...) . Please enter your OTP and confirm it with Enter/Return. The OTP is not displayed when typing. If you do not have a second factor yet, please create one (see [[Registration/2FA]]).
# The system will ask you for your service password <code>Password:</code>. Please enter it and confirm it with Enter/Return. The password is not displayed when typing. If you do not have a service password yet or have forgotten it, please create one (see [[Registration/Password]]).
# You will be greeted by the cluster with the message of the day, followed by a shell.
<pre>
$ ssh -l hd_ab123 helix.bwservices.uni-heidelberg.de
Your OTP:
Password:

********************************************************************************
* *
* Baden-Wuerttemberg HPC Cluster for *
* Structural and Systems Biology, Medical Science, *
* Soft Matter, Computational Humanities, and *
* Mathematics and Computer Science *
* *
* bwForCluster *
* _ _ _ _ *
* | | | | | | (_) *
* | |__| | ___ | | _ __ __ *
* | __ | / _ \ | | | | \ \/ / *
* | | | | | __/ | | | | > < *
* |_| |_| \___| |_| |_| /_/\_\ *
* *
* *
* (xCAT 2.16.4 / RHEL 8.8 / GPFS 5.1.8) *
* *
* *
* bwHPC Wiki : https://wiki.bwhpc.de/e/Helix *
* Ticket System : https://www.bwhpc.de/supportportal *
* Job Monitoring : https://helix-monitoring.bwservices.uni-heidelberg.de *
* E-learning : https://training.bwhpc.de/ *
* -> 'Introduction to bwForCluster Helix' *
* *
********************************************************************************

Last login: ...
[hd_ab123@login ~]$
</pre>

= Allowed Activities on Login Nodes =

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
To guarantee usability for all users you must not run your compute jobs on the login nodes.
Compute jobs must be submitted as batch jobs.
Any compute job running on the login nodes will be terminated without notice.
Long-running compilation or long-running pre- or post-processing tasks must also be submitted as batch jobs.
|}

The login nodes are the access points to the compute system, your <code>$HOME</code> directory and your workspaces.
These nodes are shared with all users. Hence, your activities on the login nodes are primarily limited to setting up your batch jobs.
Your activities may also be:
* quick compilation of program code or
* quick pre- and post-processing of results from batch jobs.

We advise to use [[Helix/Slurm#Interactive_Jobs|interactive batch jobs]] for compute and memory intensive compilation and pre- and post-processing tasks.

= Related Information =

* If you want to register a new token for the two factor authentication (2FA), consult the [[Registration/2FA|2FA Guide]].
* If you want to de-register, consult the [[Registration/Deregistration|De-registration Guide]].
* If you need an SSH key for your workflow, read [[Registration/SSH|Registering SSH Keys with your Cluster]].
* Configuring your shell: [[.bashrc Do's and Don'ts]]

= Troubleshooting =

'''Login Problems'''

If you encounter login problems, please double check the prerequisites. If those are valid, it is fastest to first check the following points before contacting the support at the [https://www.bwhpc.de/supportportal.php|bwhpc Suppport Portal].
If there are known issues with the cluster, we'll inform you via e-mail or in the message of the day at the cluster.

* Make sure to not use LAN and WLAN at the same time to prevent connection problems.
* Do [[Registration/bwForCluster/Helix#Troubleshooting_with_the_Help_of_bwServices | troubleshooting with bwServices]].

'''Lost Token'''

# To reset your token you have to (virtually) visit the IT Service at the University of Heidelberg to validate your identity. You can find the information for a virtual or in person visit in the "How to contact us" section here: https://www.urz.uni-heidelberg.de/en/support/it-service. It is recommended to use the Chrome browser for doing the video call.
# Please have an official identification ready (e.g. Führerschein / Personalausweis) and mention that you want a reset of your Helix/bwServices 2FA token. It is important to emphasize that the token reset is for bwForCluster Helix / bwServices to avoid confusion with the token that is specific to members/students of the university of Heidelberg.
#: Hint for SDS@hd+bwVisu users: All services on bwServices share the same 2FA token. If the it-service doesn't recognize "bwServices token", you can call it "Helix token" instead.
# After the IT Service deactivated the token, you can delete the token yourself, reload the page and create a new one.
Hint: To be on the safe side, it is advised to create a [[Registration/2FA#Backup_TAN_List|backup TAN list]] additionally to the smartphone token.

Helix/bwVisu/Usage

2026-09-28T09:30:06Z

H Schumacher: small correction

== Basic Navigation ==

bwVisu is quite simple and intuitively designed to make launching apps on the cluster as user-friendly as possible. Once you have successfully logged into bwVisu, the first thing you see is a list of pinned apps. These are the available apps that you can launch as a job on the cluster.

In the navigation bar, the user has the following options:

<code>Files</code> — <code>Interactive Apps</code> — <code>My Interactive Sessions</code> — <code>Shell</code>

=== Files ===

Here, you can access your home directory on Helix. In a graphical user interface, files can be downloaded and uploaded, and new files and folders can be created. You will encounter an error message if you try to see data that you have no access to.

In the Home Directory the results and logs from the apps are stored. You get the following interface:

[[File:BwVisuFiles.png|bwVisu-files|center|x300px]]

'''For SDS@hd users:''' Data on SDS@hd (under /mnt/sds-hd) can only be accessed from within jobs. Therefore, you need to open a terminal (within a JupyterLab or Rstudio job or directly on Helix) and set a symbolic link to your SDS@hd folder in your home directory. For example like this:
<pre>cd $HOME
mkdir sds-hd
cd sds-hd
ln -s /mnt/sds-hd/sd16a001 sd16a001</pre>

=== Interactive Apps ===

This displays the apps that the user can run on the cluster, possibly across multiple nodes. These apps initiate an interactive session which the user can directly connect to and, depending on the app, receive a graphical interface.

=== My Interactive Sessions ===

At bwVisu, it is possible to start several interactive apps simultaneously. Here, one can view an overview of currently running interactive apps and completed sessions. It is also possible to view the log for each session. As soon as one starts an interactive app, one is automatically redirected here. Even if you log out of bwVisu and log back in, you can still see the information and logs from the finished interactive apps here until you delete it.

=== Shell ===
For bwForCluster Helix users only: After entering your OTP and password a shell on the login node of bwForCluster Helix opens. This way, you can for example create workspaces when you need more space. Please look up the details under [[Helix/Filesystems#Workspaces | Helix Workspaces]]. They can't be seen in the File Viewer but are available within the bwVisu jobs.

== Start a Job ==
[[File:bwVisuConcept.png|center|x170px]]
<ol>
<li> At the start page of the web frontend, you can start a new job by clicking on an application version in the list of pinned apps.</li>
<li>You will now see the application description and options to adjust the job resources.</li>
<ul>
<li> To find the best resource settings for your jobs, users of the bwForCluster Helix should have a look at the [https://helix-monitoring.bwservices.uni-heidelberg.de/ bwForCluster Helix monitoring portal]. There, you can see how much of the requested resources where actually used by your jobs.
<li> At the bottom of the page, beneath the <code>Launch</code> button, you can find a link for accessing the logs/session data of all your jobs.</li>
<li>The interface can differ per app because each app can have individual configuration options. </li>
[[File:bwVisuRStudio.png|bwVisu-rstudio|center|x200px]]</li>
</ul>
<li>Click the <code>Launch</code> button to start a new job with this application. You can't start two jobs with the same application. Launching an app can take some time depending on the requested resources and cluster utilization.</li>
<li>The <code>Interactive Sessions</code> page opens with the details of your job.</li>
<ul>
<li>'''Host:''' Indicates on which nodes the application is running.</li>
<li>'''Time Remaining:''' Each application is given a time limit, which can also be set in advance. In this example, the application is supposed to run for one hour.</li>
<li>'''Session ID:''' This is a link that leads to the folder of the session. Among other things, application logs are displayed here, which can be viewed in the UI.
[[File:BwVisuRstudioRun.png|bwVisu-rstudio-run|center|x200px]] </li>
</li>
</ul>
<li> Connect to the job with your browser by clicking <code>connect to <application></code>. </li>
</ol>

== Adjust noVNC Applications ==
<div><ul>
<li style="display: inline-block; vertical-align: middle; float: right;"> [[File:bwVisu Fiji noVNCMenu.png|thumb|none|x200px|noVNC menu within Fiji]] </li>
<li style="display: inline-block; vertical-align: top; float: right;"> [[File:bwVisu_Fiji_Running.png|thumb|none|x200px|Job details view of a running Fiji Job]] </li>
</ul></div>
noVNC is an open-source VNC browser client. Some bwVisu applications use noVNC and therefore, come with additional options in the view of the job details and an additional menu within the running application.

*'''Running Job View''' on the interactive sessions page
** Compression slider: Before opening the application, you can choose the compression level to use. 0 means no compression while 9 offers the best compression but uses more CPU resources on the cluster which can be slower.
** Image Quality slider: Before opening the application, you can choose the image quality level. With this you can control the desired JPEG quality.
** View Only (Share-able Link) button: By clicking this button the application opens in view only mode. You can share this link with collaborators or students while you launch your noVNC application normally for yourself. This way, others can follow what you are doing. For every job you need to generate a new link.
* '''noVNC menu''' within the running application
** When you open the application you can see an additional menu on the left with options. For the ones using the application in view-only mode it is helpful to select the option to show a dot instead of not showing the mouse cursor at all. Therefore go to "Options" and then "Expand".
** For copying data, you need to use the clipboard. When copying something locally, you need to put it into the clipboard before you can use it within the application. The same goes the other way around.
* '''Adjust window size''' and position of the app by executing a right click at the edge of the app window frame, so that a menu shows up.

== Terminate a Job ==

The job will terminate when the job runtime requested in the job submission form has expired To terminate an application yourself, you can click the delete button or end the application in the interactive session. If the app is terminated in the interactive session, the window no longer shows <code>Running</code> but <code>Completed</code>. With the <code>Delete</code> button, you end the session and also remove the list entry from <code>My Interactive Sessions</code>.

== Troubleshooting ==
Maintance windows and known issues are communicated via the email list or the ''News'' section at the bwVisu start page in the wiki.
* Depending on the problem, it can help to look up the related bwForCluster Helix wiki pages. For example:
*: '''Error:''' Disk space exceeded → '''[https://wiki.bwhpc.de/e/Helix/Filesystems#Troubleshooting Filesystems: Troubleshooting]'''
* '''Issue:''' The webservice is not accessible.
*: If you're a bwForCluster Helix user, you need to have logged in to Helix at least once so that you're home directory gets created. Afterwards, the webservice page should be accessible.
* '''Issue:''' The bwVisu job fails or doesn't start.
*: You can try to look for the issue yourself:
*:* Reload the page without using the cache: <code>Strg+Shift+R</code> (Windows/Linux) or <code>Cmd+Shift+R</code> (Mac)
*:* Empty the browser cache:
*:** Chrome/Edge: <code>Strg+Shift+Entf</code> → choose timeframe all → choose ''Images and Files in Cache'' → choose ''Delete Files''
*:** Firefox: <code>Strg+Shift+Entf</code> → choose timeframe all → choose ''Cache'' → choose ''Delete Files''
*:* Test alternative browser (Firefox, Chrome, Edge, ...)
*:* Use a private session (without saved cookies and cache): <code>Strg+Shift+N</code> (Chrome) or <code>Strg+Shift+P</code> (Firefox).
* '''Support:''' If you can't solve the issue, write to the support and provide the following information:
** Date and time of the issue.
** Job-id (the number in brackets after the jobname)
** The <code>output.log</code> file that you can find when following the session id link.
** Please report patterns that you recognize. For example:
*** The issue happens around a specific time of day.
*** The issue is connected to the duration of usage (e.g. happens always after ca. one hour).
*** The issue appears after specific user interactions (using a specific method, opening the terminal, clicking a specific button, ...)
: You might be asked for further information:
:* The internet browser that you user and its version. For example Firefox 128.
:* Provide more information about network traffic via a HAR-file. First start a job and open the application. Then follow he instructions below:
:** Instructions for Chrome/Edge:
:**# Open developer tools with <code>F12</code> or right click -> ''Inspect'' Then choose the tab ''Network''.
:**# Activate the checkbox ''Preserve log''.
:**# Reproduce the issue in the current tab.
:**# After the problem happened again: Click the export symbol (⬇️) in the menu of the network-tabs and choose ''Export HAR...''.
:** Instructions for Firefox:
:**# Open developer tools with <code>F12</code> and choose the tab ''Network analysis''.
:**# Click the settings symbol (⚙️) and activate ''Continuous Protocol''.
:**# Reproduce the issue in the current tab.
:**# After the problem happened again: Click the settings symbol again (⚙️) and choose ''Save everything as HAR''.

Helix/bwVisu/Getting Started

2026-09-28T09:28:39Z

H Schumacher: Added troubleshooting section

== Motivation ==

When large quantities of scientific data is created (for example, from simulations), it is often necessary to employ hardware for further processing that is just as powerful as e.g. the HPC cluster that created the data in the first place. Transferring the data to a local workstation for visualization and analysis is typically not only time-consuming because of the data transfer itself, but the workstation may be ill-equipped to handle the workload. For such scenarios, remote solutions are needed, that employ powerful hardware and bring the applications to the data - and not vice versa.

bwVisu provides such a remote service. The user can log in to an intuitive web frontend which manages interactive jobs running on bwForCluster Helix. Once a job has been started, the user can connect to it such that the application appears in a browser window with full 3D hardware acceleration provided by the cluster. The only requirement on the client side is that the browser must support HTML5.

== Registration ==

In order to use bwVisu, it is required to be a user of the [https://wiki.bwhpc.de/e/Registration/bwForCluster bwForCluster Helix] or a user of [https://wiki.bwhpc.de/e/SDS@hd/Registration SDS@hd].
# Register for the service bwVisu at [http://bwservices.uni-heidelberg.de bwServices].
# bwVisu uses a 2-factor authentication (2FA) mechanism to increase security. If you have not registered a 2FA token at bwServices, an error message will appear. Use this '''[https://bwservices.uni-heidelberg.de/user/twofa.xhtml link]''' or select '''My Tokens''' in the main menu of bwServices. To register a new token, please follow these '''[[Registration/2FA|instructions]]'''. Please complete this step before continuing.
# Once your registration has been completed, your user account will be set up. This can take up to 10 minutes, so please wait for around ten minutes before proceeding.

== Login ==

# Go to the bwVisu web frontend at [https://bwvisu.bwservices.uni-heidelberg.de/ bwvisu.bwservices.uni-heidelberg.de].
# Login with the credentials of your home university and provide a 2FA token.

== Usage ==

[[File:Bwvisu2-gui.png|x150px|right|bwVisu Workflow]]
As shown in the image, the usual workflow consists of three steps: You choose an application, define the needed job resources and then connect to the job as soon as it is running.

→ '''[[Helix/bwVisu/Usage|Detailed Instructions]]'''

== Technical Information ==

=== Data storage ===

bwVisu provides no storage space. For data storage you have access to the filesystems of bwForCluster Helix and to SDS@hd.

=== Open OnDemand ===

bwVisu is based on [https://openondemand.org Open OnDemand] an open-source software platform designed to provide easy and streamlined access to high-performance computing (HPC) resources. Developed by the Ohio Supercomputer Center, it allows users to access HPC resources via a standard web browser, eliminating the need for a traditional command line interface.

The platform provides a range of tools and features. bwVisu uses it to provide the following features:
#'''Web-based Access:''' Users can manage jobs, files, and data directly through their browser.</li>
#'''Interactive Applications:'''</li>
#* It supports GUI-based applications like Jupyter Notebooks and RStudio, enabling interactive computing and visualization.</li>
#* For most bwVisu applications you can connect to a running job with your browser.</li>
#* Some applications may require a dedicated desktop client.</li>
# '''Job Management:''' Users can easily submit and manage their interactive computing jobs on HPC resources. For members of bwForCluster Helix, the jobs can be monitored via the [https://helix-monitoring.bwservices.uni-heidelberg.de/ monitoring portal].</li>
# '''File Management:''' Provides a file browser that allows users to upload and manage their data files without needing to use command-line tools.</li>

== Troubleshooting ==
In case of login problems, please follow the fitting troubleshooting guide:
* '''SDS@hd:''' [[SDS@hd/Access#Troubleshooting | Access Troubleshooting]]
* '''bwForCluster Helix:''' [[Helix/Login#Troubleshooting | Login Troubleshooting]]
* In case of '''2FA issues''', you can directly go to [[Registration/bwForCluster/Helix#Troubleshooting_with_the_Help_of_bwServices | Troubleshooting with the help of bwServices]]

Registration/bwForCluster/Helix

2026-09-28T09:13:03Z

H Schumacher: small corrections/additions

__FORCETOC__

== Prerequisites ==

Steps A + B in the [[Registration/bwForCluster | registration process]] must be finished.

== Registration Steps ==

# Visit the bwForCluster Helix '''[https://bwservices.uni-heidelberg.de registration page bwServices]'''.
#: [[File:bwServices_chooseHomeOrganisation.png|right|300px|thumb|Select your home organization]]
# '''Select your home organization''' from the list on the main page and click Proceed/Fortfahren. You will be directed to the ''Identity Provider'' of your home organization.
# Enter the '''username and password''' of your home organization (usually these credentials are also used for other services like email) and click Login/Einloggen.
# When you log in to bwServices for the first time, an overview will appear, with the account information that your home institution submits to the system. Please verify that all data is valid and then click ''Continue/Weiter''.
# You will be redirected back to the bwServices main page. It shows all the statewide services you have access to. Click '''Register/Registrieren''' under the field labeled ''bwForCluster Helix''.
#: [[File:BwIDM-reg.png|center|frame|Register for Helix]]
# bwForCluster Helix uses a '''2-factor authentication''' (2FA) mechanism to increase security.
#: If you have never registered a 2FA token on bwIDM, the following error message will appear:
#: [[File:Bwidm-3-red.png|center|600px|thumb|Second factor missing.]]
#: Use this '''[https://bwservices.uni-heidelberg.de/user/twofa.xhtml link]''' or select '''My Tokens''' in the main menu.
#: To register a new token, please follow these '''[[Registration/2FA|instructions]]'''.
#: Please complete this step before continuing.
# Read the Terms of Use / Nutzungsbedingungen, place a check mark next to ''I have read and accepted the terms of use'' and click ''Register/Registrieren''.
# '''Set a service password''' for bwForCluster Helix and click ''Save/Speichern''.
#: Be sure to use a secure password that is different from any other passwords you currently use or have used on other systems.
[[File:BwIDM-passwd.png|center|600px|thumb|Set service password]]
{|style="background:#deffee; width:70%; margin: auto;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
The service password is your private password for accessing the bwForCluster Helix. Don't ever share it with anyone!
|}

== Registration Website bwServices ==
The registration website '''[https://bwservices.uni-heidelberg.de bwServices]''' offers the following functionality:
* Register for the service.
* Set or change your service password.
* Review your registration details under "Registry Info". This includes your account status (active/lost_access).
* bwServices triggers an update of your user information upon login.
* Registering an [[Registration/SSH | SSH key]]. It can be used for login to ''bwForCluster Helix''.
* Creating tokens for logins that need 2-factor authentication ([[Registration/2FA |2FA]]) like ''bwForCluster Helix'' and ''bwVisu''.
* De-register from the service.

=== Troubleshooting with the Help of bwServices ===
When encountering connection issues with a service, the bwServices website is a good starting point for troubleshooting:
{| style="border: 2px solid #FFD28A; background-color: #ffe0af; padding: 10px; margin-bottom: 1em; width: 100%;"
|-
| '''⚠ Login to bwServices first.''' This triggers an update of your user information within some minutes and solves the connection issue in the following cases:
* User data changed: entitlement, SV/RV status, SV/RV membership status, role or email address changed or uni-id was (re)activated
* Synchronization issues of the service
|}

* <ins>'''Issue:</ins> Expired contract''' or inactive student id:
** No access to state services like the bwHPC clusters, SDS@hd or bwVisu is possible.
** You can see that this is the issue when the login to bwServices via your identity provider is not possible.
* <ins>'''Issue:</ins> Entitlement / RV / SV missing:''' Prerequisites like entitlement ownership or RV/SV membership aren't met (anymore).
** You can see the concrete issue and advice on how to move forward when looking at the registry info of the service on the bwServices website.
** When the issue is solved, the account status changes from LOST_ACCESS to ACTIVE.
* <ins>'''Issue:</ins> Service Password:''' When trying to connect to the service, the service password is not accepted.
** Set a [[#Setting_a_New_Service_Password | new password]].
* <ins>'''Issue:</ins> OTP/2FA:''' In the case of login issues with a service that uses 2FA, you can check if you're bwServices OTP is working by going in the menu to ''Index'' -> ''My Tokens''.
** If your token is not working, you might have had too many failed login attempts. You can request a reset of the error counter from the support.
** If the token needs to be reset, please follow the instructions for a [[Helix/Login#Troubleshooting | Lost Token]].
* <ins>'''Issue:</ins> Fetching Account failed:''' This issue can be detected by looking in bwServices -> Registration Details of the Service -> More Details ... -> More Properties -> "Fetching account failed" message instead of "Fetching account success". This means that there is a synchronisation issue in our system that probably causes issues for multiple users. Please report it to the support!

=== Setting a New Service Password ===

At any time, you can set a new service password via [https://bwservices.uni-heidelberg.de/ bwServices] by carrying out the following steps:
# Visit [https://bwservices.uni-heidelberg.de/ bwServices] and select your home organization.
# Authenticate yourself via your home-organizational user id / username and your home-organizational password.
# Find your service and select '''Set Service Password'''.
# Set new service password, repeat it and click the '''Save''' button. Use a secure password that is different from any other passwords you currently use or have used on other systems.
# The page answers e.g. "password has been changed".

Energy Efficient Cluster Usage

2026-09-21T13:15:14Z

H Schumacher: Added job monitoring link

Poor job efficiency means that hardware resources are wasted and a similar overall result could have been achieved using fewer hardware resources, leaving those for other jobs and reducing the queue wait time for all users.
Efficient cluster usage means choosing optimal values for job resources:
* CPU cores
* GPUs
* Memory
* (temporary) storage
* Time

__TOC__

== Motivation ==
'''User perspective'''

Short waiting times including short cycles of trial and error and fast results. The lower the job efficiency the longer the waiting times relative to the shortest possible time.

'''Energy perspective'''

Energy consumption of data centers has been increasing continuously throughout the last decade. In 2020, the energy consumption of all data centers in Germany amounted to around [https://www.bundestag.de/resource/blob/863850/423c11968fcb5c9995e9ef9090edf9e6/WD-8-070-21-pdf-data.pdf 3 percent] of the total electricity produced. Accompanying this large energy consumption are large-scale emissions of CO2 to the atmosphere and thus significant contributions to climate change.
To illustrate this, an average compute job running on a single node for one day may easily consume 10 kWh or even more. That translates roughly to brewing 700 cups of coffee.
Assuming that a typical bwHPC cluster has a few hundred compute nodes, this amounts to the energy consumption of a village for each cluster.
Although a large amount of this energy consumption is an intrinsic requirement of running large HPC clusters (even when it's processors are idle, a cluster uses a lot of energy), efficient use of the available resources is important.

Something to keep in mind:
‎<blockquote>Using as many resources as possible does not make a power user. Using them wisely does.‎</blockquote>

== Recommendations ==

* Use the '''[[Job_Monitoring | job monitoring]]''' options of your respective cluster to see the job efficiency and ways to improve it.
* '''Test''' new setups first before submitting a lot of similar jobs or resource demanding jobs.
** Run a single job first before sending many.
** Run a simplified problem on a small number of parallel entities (be it processes or threads) first before requesting many resources.
** Run a time intensive task with a short runtime first to see if it already fails in the first minutes.
* Follow the best practices of your chosen tool, software or programming language and choose the most efficient algorithms for the given problem. There are several places where you might find related documentation:
** Software Module help.
** Software page of your cluster.
** Pages linked at the [[Development]] page. For example information about debugging, performance analysis, specific programming languages etc. . → Use an efficient programming language such as Rust, C, and C++ -- well any compiled language. Do not use any interpreted language like Perl or Python. Since Machine Learning is a hot topic, this deserves a few words: Any ML-Python code using Tensorflow or other libraries will make heavy usage of NumPy and other math packages, which will use C-based implementations. Please make sure, you use the provided Python modules, which are optimized to use Intel MKL and other mathematical libraries.
Further reading:
Rui Pereira, et al: "''Energy efficiency across programming languages: how do energy, time, and memory relate?''", SLE 2017: Proc. of the 10th ACM SIGPLAN Int. Conf. on SW Language Eng., Oct. 2017, pp. 256–267, [https://doi.org/10.1145/3136014.3136031 doi:10.1145/3136014.3136031]
* If you have a task that can be scaled up, please consider a [[Scaling | '''scaling analysis''']].
* Analyse '''memory access patterns''': For small tight loops checking for locks, use the <code>pause</code> instruction.

== Things to avoid ==

* Poor choice of resources compared to the size of the nodes leaves part of the node blocked, but doing nothing:
** Too much (un-needed) memory or disk space requested
* Many small jobs with a short runtime (seconds in extreme cases). A job should run at least 10 minutes.
* When a node has 256 gb memory but only 236 gb are usable (see Hardware table of your cluster) then requesting 256 gb memory needs two nodes.
* GPU: Do not ask for a specific gpu type when your code would be fitting for any type.
* More cores used for a single mpi/openmp parallel computation than useful.
* Writing temporary files to global filesystems when a ram disk or local disk can be used.
* Do not use more parallel processes than there are actual, reasonable parallel tasks. Examples:
** 4 datasets with values for the same 1000 genes each. Method xx shall be used per gene. This operation needs 10 seconds per gene.
*** Case 1: Provide 64 single threaded cores within one job and use one process/thread per gene: This creates a large overhead as only 64 processes can run in parallel while all 5000 processes try to get computing time so that a process might run for 5 seconds and then makes room for another process that also isn't able to finish because another process comes in between.
*** Case 2: Send 1 job per gene so that each job uses method xx on this gene in all 4 datasets: This creates a large overhead as each job only needs 50 seconds while there is a thousand times the overhead of creating and finishing a job.
*** Case 3: Provide 64 cores in one job. Each dataset uses 16 cores. Each core needs to process 1000/16 =~ 63 genes. This needs 63*10 seconds
= 10.5 minutes job runtime with just the overhead of creating a single job and 16 processes without switching between processes.
** → Simple parallelization by hand is advisable. See: A basic introduction to [[Development/Parallel_Programming | Parallel Programming]].

== Fair Share and Scheduler ==
Influence of Efficiency on Fair Share and Scheduler:
* '''Fair Share value:'''
*: The fair share value represents your priority on the cluster. On a busy cluster a low priority leads to longer waiting times as other users with a higher priority have a higher chance of getting free resource slots. The fair share value declines when a lot of cluster resources were used recently. Used is defined as follows:
*:* Requested resources: 4 gpus for 2 hours
*:* Actual resource usage: 1 gpu and job finishes successfully after 1 hour.
*:* Used resources as considered by the fair share value: 4 gpus for 1 hour because the three idle gpus couldn't be used by any other cluster used as they were blocked by this job.
* '''Job scheduler:'''
*: The longer the requested runtime and the more resources are needed in that time the harder it is to find a fitting timeframe where these resources are free.

Job Monitoring

2026-09-21T13:13:59Z

H Schumacher: created page

This page provides links to the job monitoring documentation for each cluster:

* [[BwUniCluster3.0/Running_Jobs#Monitor_and_manage_jobs | bwUniCluster]]
* [[BinAC2/Slurm#Detailed_job_information_:_scontrol_show_job | BinAC]]
* [[Helix/Slurm#Job_Monitoring | Helix]]
* [[JUSTUS2/Jobscripts:_Running_Your_Calculations#Monitoring_Your_Jobs | JUSTUS]]
* [[NEMO2/Slurm#Job_Monitoring | NEMO]]

Development/VS Code

2026-09-11T16:21:17Z

H Schumacher: Improved echo output. Made the simple run script usable via . ./script.sh as well

== Overview ==

[[File:vscode.png|thumb|Visual Studio Code, Source: https://code.visualstudio.com/|450px]]

[https://github.com/Microsoft/vscode Visual Studio Code] (VS Code) is an open source source-code editor from Microsoft. It has become one of the most popular IDEs according to a [https://survey.stackoverflow.co/2024/technology#1-integrated-development-environment stackoverflow survey]. The functionality of VS Code can easily be extended by installing extensions. These extensions allow for almost arbitrary '''language support''', '''debugging''' or '''remote development'''. You can install VS Code locally and use it for remote development.

== Visual Studio Code ==
Visual Studio Code (VS Code) is a lightweight, extensible code editor from Microsoft that supports many programming languages and features like debugging, and integrated Git. It offers a rich extension marketplace to add language support, themes, and tools tailored to your workflow. VS Code runs on Windows, macOS, and Linux and is popular for its speed, customizability, and strong community ecosystem.

=== Using AI agents ===
When deploying AI agents on the bwHPC clusters, users must exercise extreme caution and maintain full oversight of the agent's activities. You are fully responsible for all actions initiated by an agent, including any security breaches or system disruptions it may cause. It is mandatory to strictly monitor resource usage on login nodes, as these are shared resources intended only for lightweight tasks. Any agent found consuming excessive CPU or memory on a login node will be terminated immediately to ensure stability for other users. To comply with usage policies, all AI-driven workloads which generate heavy load must be submitted to the Slurm batch queues rather than running directly on the login nodes.

=== Extension: Remote-SSH ===

In order to remotely develop and debug code at HPC facilities, you can use the [https://code.visualstudio.com/docs/remote/ssh '''Remote - SSH''' extension]. The extension allows you to connect your locally installed VS Code with the remote servers. So in contrast to using graphical IDEs within a remote desktop session (RDP, VNC), there are no negative effects like e.g. laggy reactions to your input or blurred display of fonts.

==== Installation and Configuration ====

[[File:vscode-extensions-button.png|vscode-extensions-button.png|30px]] 
In order to install the Remote - SSH extension, just click on the Extensions (Erweiterungen) button in the left side bar and enter “remote ssh” in the search field. Choose '''Remote - SSH''' from the occurring list and click on '''Install'''.
 
[[File:vscode-remoteexplorer-button.png|vscode-remoteexplorer-button.png|30px]] 
In order to configure remote connections, open the Remote-Explorer extension. On Linux Systems, the file <code>~/.ssh/config</code> is automatically evaluated. The targets within this file already appear in the left side bar.
 
[[File:vscode-remoteexplorer-add.png|vscode-remoteexplorer-add.png|350px]] 
If there are no remote ssh targets defined within this file, you can easily add one by clicking on the + symbol. Make sure that “SSH Targets” is active in the drop down menu of the Remote-Explorer. Enter the connection details <code><user>@<server></code>. You will be asked, whether the file <code>~/.ssh/config</code> should be modified or if another config file should be used or created.

A minimal entry within <code>~/.ssh/config</code> in order to have a remote target to be listed there could look like this:

$ cat ~/.ssh/config
Host uc3.scc.kit.edu
HostName uc3.scc.kit.edu
User xy_ab1234

==== Connect to Login Nodes ====

[[File:vscode-remoteexplorer-button.png|vscode-remoteexplorer-button.png|30px]] 
In order to connect to a remote SSH target, open the Remote-Explorer. Right-click a target and connect in the current or a new window. TOTP and password can be entered in the corresponding input fields that open.

You are now logged in on the remote server. As usual, you can open a project directory with the standard key binding Ctrl+k Ctrl+o. You can now edit and debug code.

'''Attention''': Please remember that you are running and debugging the code on a login node. Do not perform resource-intensive tasks. Furthermore, no GPU resources are available to you.

Extensions, which are installed locally, are only usable on your local machine and are not automatically installed remotely. However, as soon as you open the Extensions-Explorer during a remote session, VS Code proposes to install the locally installed extensions remotely.

==== Disconnect from Login Nodes ====

[[File:vscode-remoteexplorer-indicator.png|images/vscode-remoteexplorer-indicator.png|200px]] 
If you want to end your remote session, click the green box in the lower left corner. In the input box that opens, select the “Close Remote Connection” option. If you simply close your VS Code window, some server-side components of VS Code will continue to run remotely.

If you encounter any issues with the remote connection or if you didn't end the sessions correctly, it can help to kill the VS Code Server on the host. This is done by going to the Command Palette (via the 'Manage' wheel in the lower left corner) and executing the following command:

<code> Remote-SSH: Kill VS Code Server on Host </code>

=== Access to Compute Nodes ===

The workflow described above does not allow debugging on compute nodes that have been requested via an interactive Slurm job, for example. Debugging GPU codes is therefore also not possible, since this kind of resource is only accessible within Slurm jobs.
We strongly discourage using the Code Tunnel application, as it violates our access policies. In this scenario, an application running on the compute node connects to a Microsoft or GitHub server. The locally running VS Code then connects to the compute nodes via these external servers, thereby bypassing the login nodes.



== Code-Server ==

The application [https://github.com/cdr/code-server code-server] allows to run the server part of VS Code on any machine, it can be accessed in the web browser running on your local computer. This enables, for example, development and debugging on compute nodes.
Code-server runs a web server which serves on an unprivileged port. In order to connect your web browser to the remotely running code server, you have to forward this port via a SSH tunnel.

[[File:code-server.png|thumb|code-server.png|VS Code in web browser: code-server, Source: https://github.com/cdr/code-server">https://github.com/cdr/code-server|400px]]

=== Install Code-Server ===

Code-server is pre-installed on bwUniCluster and accessible via an Lmod module:
<source lang="bash">module load devel/code-server</source>

On clusters with no code-server module, the application can easily be installed with the description available on the official [https://github.com/coder/code-server GitHub page].

=== Start Code-Server ===

Code-server can be run on either login nodes or compute nodes. In the example shown, an interactive job is started on a GPU partition to run code-server there.

<syntaxhighlight lang="console">$ salloc -p accelerated --gres=gpu:4 --time=30:00 # Start interactive job with 1 GPU
$ module load devel/code-server # Load code-server module</syntaxhighlight>
When code-server is started, it opens a web server listening on a certain port. The user has to '''specify the port'''. It can be chosen freely in the unprivileged range (above 1024). If a port is already assigned, e.g. because several users choose the same port, another port must be chosen.

By starting code-server, you are running a web server that can be accessed by anyone logged in to the cluster. To prevent other people from gaining access to your account and data, this web server is '''password protected'''. If no variable <code>PASSWORD</code> is defined, the password in the default config file <code>~/.config/code-server/config.yaml</code> is used. If you want to define your own password, you can either change it in the config file or export the variable <code>PASSWORD</code>.

<syntaxhighlight lang="console">$ PASSWORD=<mySecret> \
code-server \
--bind-addr 0.0.0.0:8081 \
--auth password # Start code-server on port 8081</syntaxhighlight>

{| style="background:#FFCCCC; width:100%;"
| '''Security implications'''
Please note that by starting <code>code-server</code> you are running a web server that can be accessed by everyone logged in on the cluster. 
* '''If password protection is disabled, anybody can access your account and your data.'''
* Choose a '''secure password'''!
* Do '''NOT''' use <code>code-server --link</code>!
|}

=== Connect to code-server ===
[[File:code-server-hk.png|thumb|Code-server running on GPU node.|400px]]

As soon as code-server is running, it can be accessed in the web browser. In order to establish the connection, a SSH tunnel from your local computer to the remote server has to be created via:

<syntaxhighlight lang="console">$ ssh -L 8081:<computeNodeID>:8081 <userID>@<hostname></syntaxhighlight>
You need to enter the <code>computeNodeID</code> of the node on which the interactive Slurm job is running. If you have started code server on a login node, just enter <code>localhost</code>. Now you can open http://127.0.0.1:8081 in your web browser. Possibly, you have to allow your browser to open an insecure (non-https) site. The login site looks as follows:

[[File:code-server-login.png|Code-server login page.|300px]]

Enter the password from <code>~/.config/code-server/config.yaml</code> or from the <code>PASSWORD</code> variable. After clicking the “Submit” button, the familiar VS Code interface will open in your browser.

=== End code-server session ===

If you want to temporarily log out from your code-server session you can open the “Application Menu” in the left side bar and click on “Log out”. To '''terminate''' the code-server session, you have to cancel it in the interactive Slurm job by pressing ++ctrl+c++.

== Connect to Remote Jupyter Kernel ==

To work with your python scripts and notebooks within VSCode (or JupyterLab) while using the resources of a compute node, you can create a batch job that launches JupyterLab and connect to it via VS Code or via an URL. To do so, please follow the instructions below. Any parts of the scripts that might need adjustments are marked with the keyword "@params".

=== Simple Use Case ===
The most basic steps are to set a password for JupyterLab, start a job which runs JupyterLab, get the connection details from the output log and connect to it locally. The following instructions explain these steps and provide an additional script that replaces the manual step of looking into the output file.

# Load a python module and set a password on the cluster for JupyterLab:
#: <syntaxhighlight lang="bash">
module load devel/miniforge
jupyter notebook --generate-config
jupyter notebook password
</syntaxhighlight>
# Define a batch script to start a JupyterLab Job. Please adjust the first part according to your needs and your specific cluster.
#: <pre>~/jupyterlab.slurm</pre>
#: <syntaxhighlight lang="bash">
#!/bin/bash

#SBATCH --partition=cpu-single
#SBATCH --job-name=jupyterlab
#SBATCH --time=00:10:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task 1
#SBATCH --mail-user=my_email_address #my_email_address # to use this generic version, add "alias my_email_address=<yourEmailAddress>" to the ~/.bashrc file
#SBATCH --mail-type=ALL

# @param: change this to your preferred python or conda module
module load devel/miniforge

# @param: cluster address for ssh connection
hostAddress=helix.bwservices.uni-heidelberg.de

PORT=$(( ( RANDOM % 9999 ) + 1024 ))
# Extract from the squeue output all jobs containing 'jupyterlab' and take the first entry in the node column.
HOSTID=$(squeue -h -o "%A %N %j" | grep jupyterlab | awk 'NR==1{print $2}')
echo "Job {$SLURM_JOB_ID} is running on host {$HOSTID}."
echo "Connect by typing into your local terminal:"
echo "ssh -N -L ${PORT}:${HOSTID}:${PORT} ${USER}@$hostAddress"
jupyter lab --no-browser --ip=0.0.0.0 --port=${PORT}

returned_code=$?
echo "> Script completed with exit code ${returned_code}"
exit ${returned_code}
</syntaxhighlight>
# Run a wrapper script to execute the batch script and extract needed information from the slurm output file. You could save it together with other utility scripts in a "bin" directory in your home folder.
#: <pre>./bin/run_jupyterlab_simple.sh</pre>
#: <syntaxhighlight lang="bash">

#!/bin/bash

# Define parameters
url=""
jobscript=~/jupyterlab.slurm
hostAddress=helix.bwservices.uni-heidelberg.de

# Run job
job_id=$(sbatch $jobscript | awk '{print $4}')
echo "jobid: $job_id"

# Outfile name
slurm_out=slurm-${job_id}.out

# Wait for output file
while [ ! -f $slurm_out ]; do
sleep 2;
done

# Wait until url is written in output file
while [ -z ${url} ]; do
sleep 1;
url=$(grep -o 'http[^ ]*' $slurm_out | head -n 1);
done

# Extract hostID and port from output. The pattern assumes a node name with a length of 6 characters and a port with a length of 3, 4 or 5 numbers.
url_pattern="http://([a-z0-9]{6}):([0-9]{3,5})/lab"
if [[ $url =~ $url_pattern ]]; then
hostID=${BASH_REMATCH[1]}
port=${BASH_REMATCH[2]}
echo "To connect with the JupyterLab kernel, please enter the following into your local commandline: "
echo "ssh -N -L $port:$hostID:$port ${USER}@$hostAddress";
echo ""
echo "Note: It is normal that the ssh command doesn't end after providing the credentials. Ending the command would mean ending the local connection to the kernel."
echo ""
echo "Afterwards, you can use the URL"
echo " http://127.0.0.1:${port}/lab "
echo ""
echo "to:"
echo "- use the kernel in VSCode ('Existing Jupyter Server...', enter URL, enter password, confirm '127.0.0.1', choose kernel) or "
echo "- open JupyterLab in your browser with the URL"
else
echo "The needed information couldn't be found in the slurm output. Please contact your support unit if you need help with fixing this problem."
fi
# rm $slurm_out
</syntaxhighlight>
# Follow the instructions on the commandline to connect to the Jupyter kernel from your local machine or the Helix login node. More detailed instructions can be found below.

==== Connect to a running job ====

The job runs on a specific compute node and port. With this information, you can create a ssh connection to it. But first, you need to decide, in which way you want to work with your python code. The options are:

# The code is placed locally on your computer.
# The code is placed on the cluster and you've mounted the folder locally. (= The files on the cluster are accessible from within your local VS Code)
# The code is placed on the cluster and you work on the cluster via a remote connection in VS Code.

Depending on the use case, you need to execute the ssh command in a different place:

# Open VS Code on your computer.
# Open VS Code on your computer.
# Open VS Code on your computer and connect to the cluster.

Then open a terminal and execute the ssh command, which is given in the commandline output of the wrapper script. If the terminal isn't already open, go to menu item "Terminal" at the top of the window and choose "New Terminal" (or "new -> command prompt" on Windows).
It is normal that the command doesn't end after you've put in your credentials. Leave the terminal open and go on with the next step.

To use the jupyter kernel that is running on the cluster node, you need to connect this kernel. This is similar to connecting any other kernel:

# Open your code file.
# Click "Select Kernel" in the upper right corner.
# Choose "Existing Jupyter Server...".
# Enter the URL that was given by the wrapper script.
# Enter your JupyterLab password that you set in the first step of these instructions.
# Confirm the prefilled value "127.0.0.1" by pressing Enter.
# Choose one of the virtual environments that you've created on the cluster. You should see all python environments. To see the conda environments as well, you need to [[Helix/bwVisu/JupyterLab#Python_version | register them as ipykernel]] first.

=== Complex Use Case ===
If you have different use cases for juypterlab, you could use a more flexible wrapper script, for example:

<pre>./bin/run_jupyterlab.sh</pre>

<syntaxhighlight lang="bash">
#!/bin/bash
# Starts a jupyter kernel on a node and provides information on how to connect to it locally.
# If you have only one use case and therefore need only one combination of slurm settings for your jupyter jobs, then you can use the simpler script.
# This script supports explorative analyses by allowing to overwrite parameters via commandline.
# Different job configurations can be defined in advance and then used with a given short name (cpu, gpu,...).

programname=$0
function help {
'''help text'''
echo ""
echo "Starts a jupyterlab kernel"
echo ""
echo "usage example: $programname --param_set cpu"
echo ""
echo " --param_set string name of the parameter set"
echo " (examples: cpu, gpu)"
echo " --jobscript string optional, path of batch script"
echo " (default: ~/jupyterlab.slurm)"
echo " --slurm_out string optional, name of slurm output file"
echo " (default: slurm-${job_id}.out)"
}

# These parameters are set later in the script. Providing them via commandline, overwrites their values set in the script.
jobscript=None
slurm_out=None

# Process parameters
while [ $# -gt 0 ]; do
if [[ $1 == "--help" ]]; then
help
exit 0
# when given -p as parameter, use its value for the variable param_set
elif [[ $1 == "-p" ]]; then
param_set="$2"
shift
elif [[ $1 == "--"* ]]; then
v="${1/--/}"
declare "$v"="$2"
shift
fi
shift
done

function define_param_set(){
'''Define parameter sets for sbatch'''
# Define different sets
cpu=(--partition=cpu-single --mem=2gb)
gpu=(--partition=gpu-single --mem=3gb --gres=gpu:1)

param_set=${1}
param_set=$param_set[@]
param_set=("${!param_set}")

# Add params that are the same for all sets
param_set+=(--ntasks=1)
}

# @param: jobscript, name of the slurm batch script to execute
if [ "$jobscript" = "None" ]; then
jobscript=~/jupyterlab.slurm
fi

# @param: cluster address for ssh connection
hostAddress=helix.bwservices.uni-heidelberg.de

# Translate given param_set value to actual set of parameters
define_param_set $param_set
echo "param_set: ${param_set[*]}"

# Run job
job_id=$(sbatch ${param_set[@]} $jobscript | awk '{print $4}')
echo "jobid: $job_id"

# @param: slurm_out, the filename for the slurm output file
if [ "$slurm_out" = "None" ]; then
slurm_out=slurm-${job_id}.out
fi

# Wait for output file
while [ ! -f $slurm_out ]; do
sleep 1;
done

# Wait until url is written in output file
while [ -z ${url} ]; do
sleep 1;
url=$(grep -o 'http[^ ]*' $slurm_out | head -n 1);
done

# Extract hostID and port from output.
url_pattern="http://([a-z0-9]{6}):([0-9]{3,5})/lab"
if [[ $url =~ $url_pattern ]]; then
hostID=${BASH_REMATCH[1]}
port=${BASH_REMATCH[2]}
echo "To connect with the JupyterLab kernel, please enter the following into your local commandline: "
echo "ssh -N -L $port:$hostID:$port ${USER}@$hostAddress";
fi

echo "Afterwards, you can either"
echo "- use the kernel in VSCode or "
echo "- open JupyterLab with this URL: "
echo " http://127.0.0.1:${port}/lab "
echo "Note: It is normal that the ssh command doesn't end after providing the credentials. Ending the command would mean ending the local connection to the kernel."
#rm $slurm_out
</syntaxhighlight>

Helix/Software

2026-09-10T15:22:38Z

H Schumacher: Added mention of Alphafold and Link to BioStructureHub Tutorials

== Environment Modules ==
Most software is provided as [[Environment Modules]].

== Available Software ==
Visit [https://www.bwhpc.de/software.php https://www.bwhpc.de/software.php], select <code>Cluster → bwForCluster Helix</code>

On the cluster: <code>module avail</code>

== Documentation on the Cluster ==
Documentation for each environment module is available on the cluster:
* with command <code>module help</code>
* examples are in <code>$<SOFTNAME>_EXA_DIR</code> (batch script + other needed input files). This environment variable is available as soon as the module is loaded.
** Some modules, e.g. the Alphafold module, include an especially extensive example.

== Documentation in the Wiki ==
For some environment modules additional documentation is provided here.

* [[Helix/Software/Singularity | Singularity]] (containers)
* [[Development/Conda | Conda]]
* [[Python | Python]]
* [[Helix/Software/R | R]]
** [[Helix/Software/R/Rstan | Rstan]]
* [[Helix/Software/Matlab | Matlab]]
* [[Helix/Software/SAS | SAS ]]
* [[Helix/Software/Stata | Stata]]

== External Documentation ==

* '''BioStructureHub Heidelberg:''' Tutorials are provided for using Helix software modules like Alphafold, RFDiffusion etc. and related software on the Helix cluster and on [[Helix/bwVisu | bwVisu]].
*: → [https://github.com/ssciwr/BioStructureHub/tree/main/docs/tutorials Tutorials]

Registration/bwForCluster/RV

2026-09-07T14:50:45Z

H Schumacher: Added Cluster Overview Section

'''Step B: Apply for a Rechenvorhaben/Project'''
__TOC__
Compute Projects (Rechenvorhaben / '''RVs''') are required for assignment to a bwForCluster.
[[File:RV-my.png|right|frame|My RVs.]]

To get access to an RV, answer the following question:
* Does your work group (or the collaborative project that you are part of) already have an RV?
** '''Yes → [[#Become_Coworker_of_an_RV|Become Coworker of an RV]]'''
** '''No → [[#Register_a_new_RV|Register a new RV]]'''

You can see your RV memberships at the central application site ZAS under [https://zas.bwhpc.de/shib/en/info_rv.php '''My RVs'''].

== Become Coworker of an RV ==

[[File:RV-collab.png|right|100px|frame|1. RV collaboration.]]
[[File:RV-formcol.png|right|300px|thumb|4. RV collaboration form]]
{|style="background:#deffee; width:70%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
You can only sign up for an RV once.
If you are an inactive member of an RV, either because you de-registered yourself or a manager deactivated you, only an RV manager can reactivate you.
|}

The "RV responsible" will provide you with the following data on the RV:
* RV acronym
* RV password

If this information is lost, please let the "RV responsible" [[Registration/bwForCluster/RV/Management#Renew_RV_Password|renew the password]] for the RV.

To become coworker of an RV, follow these steps:
# Please visit the [https://zas.bwhpc.de/shib/en/bwforcluster_collaboration.php '''RV collaboration'''] web form at the [https://zas.bwhpc.de ZAS].
# Select your home organization from the list on the main page and click '''Proceed''' or '''Fortfahren'''.
# You will be directed to the ''Identity Provider'' of your home organization. Enter the username and password of your '''home organization''' (usually these credentials are also used for other services like email) and click '''Login/Einloggen'''.
# You will be redirected back to the RV collaboration form '''https://zas.bwhpc.de/shib/en/bwforcluster_collaboration.php'''.
# Enter '''RV Acronym''', '''RV Password''' and '''Institute''' into the form and check both "Terms of Use". Click '''Send'''.
# You will be assigned to the RV as a new member.
<div style="padding-left:18px;">
[[Image:Attention.svg|left|15px]]
In case of the error "This project is not currently renewed", please ask the RV responsible to apply for a prolongation of the RV. As soon as the prolongation was accepted, you can join the RV.
</div>

After submitting the request you will receive an email from ZAS about the further steps.
RV owner and RV managers will be notified automatically.

== Register a new RV ==

{|style="background:#deffee; width:100%;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
An RV is valid for '''one year'''.
After that, the RV responsible must apply for an extension of the RV for it to be renewed for another year.
|}

An RV should be registered by a leader or senior scientist of a research group/collaboration.
You will be:
* held accountable for the coworkers in the RV
* asked to provide information for the two reports required by the DFG for their funding of bwForClusters
* likely asked for a contribution to a future DFG grant proposal for a new bwForCluster in your area of research ("wissenschaftliches Beiblatt")

'''Some hints:'''
* To be notified about important issues via the cluster's mailing list, it is necessary to finish all three registration steps. If you don't plan to work with the cluster yourself, we recommend to either register nevertheless or ask a colleague who is going to use the cluster to keep you updated. For better transparency, that colleague could receive the ''RV manager'' role from you.
* Any number of coworkers can join the RV to work with the cluster. Therefore, they need to be provided with the RV password and acronym.
* The [https://www.bwhpc.de/bwhpc-domains.php research area] of the group as well as the software and hardware requirements are taken into account.
* Creating mutiple RVs on the same cluster is not necessary. Being part of multiple RVs on the same cluster does not bring any advantages regarding available computing resources.
* Getting access to two different clusters is only possible if specific software or hardware needs demand access for both.

[[File:RV-reg.png|right|frame|Register a new RV.]]
'''To apply for a new RV, follow these steps:'''

# Please visit the [https://zas.bwhpc.de/shib/en/bwforcluster_proposal.php '''RV registration'''] web form at the [https://zas.bwhpc.de ZAS]. Select your home organization, then enter the credentials for your home organization.
# Fill in the form by following the advice in the table below.
#* Hint: The answers won't be restrictive in any way. The RV description and resource specifications are important for finding the best fitting cluster. But they don't set any limits. Every cluster user has full access to all cluster resources. And not all research topics need to match the RV description.
#* Hint: You can switch to the [https://zas.bwhpc.de/shib/bwforcluster_proposal.php German version] of the web form.

{| style="width:100%;border:3px solid darkgray; margin: 1em auto 1em auto;"
|-
! style="width:20%;padding:2px;margin:3px; background:#AAA; font-size:100%; font-weight:bold; border:1px solid #BBBBBB; text-align:left; color:#000; padding:0.2em 0.4em;"| Field
! style="padding:2px;margin:3px; background:#AAA; font-size:100%; font-weight:bold; border:1px solid #BBBBBB; text-align:left; color:#000; padding:0.2em 0.4em;" | Explanation
|rowspan="15"| [[File:RV-form.png|center|400px|thumb|Registering a new RV.]]
|-
!scope="column" {{Lightgray_small}}| RV Title
|scope="column" {{Tab element}} | Define a short title of your planned compute activities, maximum of 255 characters including spaces.
|-
!scope="column" {{Lightgray_small}}| RV Description
|scope="column" {{Tab element}} | Write a short abstract about your planned compute activities (maximum of 2048 characters including spaces). Research groups that contributed to the DFG grant proposal (Art. 91b GG) of the corresponding bwForCluster only need to give reference to that particular proposal. Please mention in your description for which research group or collaborative project you're opening the RV.
|-
!scope="column" {{Lightgray_small}}| Scientific field
|scope="column" {{Tab element}} | Please look carefully through the scientific fields listed in the [https://www.dfg.de/resource/blob/175334/89ba4a3464c99aaea40fdef47367e7b2/fachsystematik-2020-2024-de-grafik-data.pdf DFG classification] to find the field that fits best to your research focus and experience. This way, you'll be assigned to the cluster that can support you best. Make sure to choose the most specific field. If multiple scientific fields are matching, add all of them in the text box. If none are matching, state your field in the textbox.
|-
!scope="column" {{Lightgray_small}}| Domain
|scope="column" {{Tab element}} | Be aware that all clusters offer specialized support for all computational activities within their assigned scientific fields. Please provide a reason in your description if you change the preselected domain/cluster to another one. If the domain "Other" is preselected, please read the [[Registration/RV#Cluster_Overview | cluster overview]] thoroughly before deciding on one of them. If you'd like more guidance, keep the domain "Other" and we will come back to you.
|-
!scope="column" {{Lightgray_small}}| Field of activity
|scope="column" {{Tab element}} | Define if your RV is primarily for research and/or teaching. If not applicable, use text box.
|-
!scope="column" {{Lightgray_small}}| Parallel paradigm
|scope="column" {{Tab element}} | State what parallel paradigm your code or application uses. Multiple ticks are allowed. Further information can be provided via text box. If you are not sure about it, please state the software you are using.
|-
!scope="column" {{Lightgray_small}}| Programming language
|scope="column" {{Tab element}} | State the programming language(s) of your code or application.
|-
!scope="column" {{Lightgray_small}}| Numerical methods
|scope="column" {{Tab element}} | State the numerical or "calculation" methods your code or application utilizes. If you do not know, write "unknown".
|-
!scope="column" {{Lightgray_small}}| Software packages
|scope="column" {{Tab element}} | State all software packages that your group/project plans to use for your RV. Also include compiler, MPI and numerical packages (e.g. MKL, FFTW).
|-
!scope="column" {{Lightgray_small}}| Estimated requested computing resources
|scope="column" {{Tab element}} | Roughly estimate how many CPU hours are required to finish your RV. To calculate "CPU hours" multiply "elapsed parallel computing time" with "number of CPU core" per job.
''Example: Your code uses 4 CPU cores and has a wall time of 1h. This makes 4 CPU hours.''
|-
!scope="column" {{Lightgray_small}}| Planned maximum number of parallel used CPU cores per job
|scope="column" {{Tab element}} | Give an upper limit of how many CPU cores per job are required. Please cross-check with your statement about parallel paradigm.
''Obviously, a sequential code can only use 1 CPU core per job.''
|-
!scope="column" {{Lightgray_small}}| Estimated maximal memory requirements per CPU core
|scope="column" {{Tab element}} | Give an upper limit of how much RAM per CPU core your jobs require. Please give a value in Gigabytes.
|-
!scope="column" {{Lightgray_small}}| Estimated requested persistent disk space for the RV
|scope="column" {{Tab element}} | State how much disk space in total you need for your RV. Please give a value in Gigabytes.
''Example: If your RV has 4 more coworker and each of you produces until the end of your RV 20 Gigabyte of output, your maximum disk space is 100 Gigabyte.''
|-
!scope="column" {{Lightgray_small}}| Estimated maximal temporary disk space per job
|scope="column" {{Tab element}} | State how much space your intermediate files require during a job run. Please give a value in Gigabytes.
''Example: Your final output of your job is 0.1 Gigabyte, but during your job 10 temporary files, each with a size of 1 Gigabyte, are written to disk, hence your correct answer is 10 Gigabyte.''
|-
!scope="column" {{Lightgray_small}}| Own Personal Data (RV responsible)
|scope="column" {{Tab element}} | Some fields are auto-filled and can not be changed. These are your credentials as given by your home organization.
|-
!scope="column" {{Lightgray_small}}| Institute
|scope="column" {{Tab element}} | State the full name of your institute.
|-
|}
<ol start="3">
<li>Click '''Preview''' and then '''Submit'''.</li>
<li>The submitter of the RV application will be assigned the role "RV responsible" (see [[Registration/bwForCluster/RV/Management|Roles in an RV]]).</li>
<li>Go to section [[Registration/bwForCluster/RV#Registration_Response|Registration Response]] for the next steps.</li>
</ol>

=== Cluster Overview ===
This chapter provides an overview of different characteristics that the four clusters have to offer.

<ins>Software in general:</ins>
* → [https://www.bwhpc.de/software.html software modules per cluster].
* Additional software of bwForCluster Helix can be found [[Helix/bwVisu | here]].
* All clusters are willing to add needed software modules or alternatively help with software installations.
----
[[File:JUSTUS2_header.png|100px]]

The '''[[JUSTUS2 | bwForCluster JUSTUS]]''' is dedicated to research in the following bwHPC domains:
* Theoretical Chemistry
* Condensed matter physics
* Quantum physics

The cluster offers for example:
* Basic license of the Schrödinger-Suite, Gaussian
* Remote Visualization via [[JUSTUS2/Visualization | VNC]]
* External Storage: [[SDS@hd | SDS@hd]] can be mounted with Rclone.
* → [[JUSTUS2/Hardware | see available Hardware]]
----
[[File:Serverraum-2024-09.jpg|left|50px]]

The '''[[NEMO2 | bwForCluster NEMO]]''' is dedicated to research in the following bwHPC domains:
* Neurosciences
* Particle physics
* Materials science
* Micro system engineering

The cluster offers for example:
* Remote Visualization with [[JUSTUS2/Visualization | VNC]]
* External Storage: [[SDS@hd | SDS@hd]] can be mounted with Rclone.
* → [[NEMO2/Hardware | see available Hardware]]
----
[[File:Helix-logo.png|100px]]

The '''[[Helix | bwForCluster Helix]]''' is dedicated to research within the following bwHPC domains:
* Structural and Systems Biology & Medical Sciences. Some examples:
** Usage of bioinformatic tools and tool development
** Data analysis (Omics, Cryo-EM, ...)
** Pipeline development with e.g. snakemake, nextflow
* Soft Matter
* Computational Humanities
* Mathematics and Computer Science
[[File:bwvisu2-gui.png|250px|thumb|bwVisu: Steps to start a JupyterLab job.]]
The cluster offers for example:
* Remote Visualization with [[File:BwVisu wide.svg|80px]]: Easy to use web-based, interactive access to various software (JupyterLab, RStudio, Fiji, ...). No terminal needed to send jobs. → [[Helix/bwVisu | bwVisu]]
* External Storage [[File:sds-hd-logo.png|80px]]: Direct access to [[SDS@hd | SDS@hd]]. Very fast in case of i/o heavy operations as the cluster and storage are both hosted in Heidelberg.
* [[Helix/Slurm#Job_Monitoring_Portal | Job Monitoring]]: A website is offered to see and improve job efficiency for fast jobs and low waiting times.
* → [[Helix/Hardware#Compute_Nodes | see available Hardware]]
----
[[File:BinAC2_Logo_RGB_subtitel.svg|80px]]

The '''[[BinAC2 | bwForCluster BinAC]]''' is dedicated to research in the following bwHPC domains:
* Bioinformatics & Medical Informatics
* Pharmacy
* Astrophysics
* Geosciences
The cluster offers for example:
* Remote Visualization with [[BinAC2/Software/TigerVNC | TigerVNC]]
* External Storage: Direct access to [[SDS@hd | SDS@hd]] after initial setup.
* A [[BinAC2/Software/Jupyterlab | JupyterLab software module]] is available.
* → [[BinAC2/Hardware_and_Architecture | see available Hardware]]
----

=== Registration Response ===

After submitting you will get an email from the ZAS confirming your submission.
With this email you are given an unique
* RV acronym
* RV password

Please keep the password enclosed.
With RV acronym and password, [[Registration/bwForCluster/RV#Become_Coworker_of_a_RV| coworkers ]] can already associate themselves to the RV.
If this information is lost, please [[Registration/bwForCluster/RV#Renew_RV_Password|renew the password]] for the RV.

The cluster assignment team assigns a bwForCluster to your RV based on your submitted data, including the [https://www.bwhpc.de/bwhpc-domains.php research area] as well as the software and hardware requirements.
The ZAS notifies you in an email about your assigned bwForCluster and provides website details for [[Registration/bwForCluster/Service| Step C]]. Later on you can find details about the roles and management possibilities at the [[Registration/bwForCluster/RV/Management | RV Management]] page.

=== Duration of an RV ===

An RV is valid for '''one year'''.
After that, the RV responsible must apply for an extension of the RV for it to be renewed for another year. We appreciate it if you update the RV information along with the prolongation application.

=== RV Management for RV Responsibles and Managers ===

RV responsible or manager can [[Registration/bwForCluster/RV/Management|manage and modify Rechenvorhaben/projects]].

----

[[Registration/bwForCluster/Service | Go to step C]]

SDS@hd/Registration

2026-09-04T11:01:51Z

H Schumacher: improved explanation

== Prerequisites ==

* Only scientific data is stored. Don't place administrative data on SDS@hd.
* Only hot data is stored which means only data that is actively used is allowed. We expect you to remove data from finished projects. It is advised to use an archiving service for this. Temporary data should also be removed on a regular basis.
* The storage of personal data or data capable of being linked to an individual is not permitted.
* SDS@hd mustn't be used as an archive or a backup. To find solutions in this regard, please have a look at the [https://wiki.bwhpc.de/e/Main_Page Main Page] and your institutional services.
* The use of SDS@hd must be adequately documented:
** All publications that were created with the involvement of the storage service SDS@hd should acknowledge its use. For an example text see [[SDS@hd/Acknowledgement | here]].
** These publications, scientific posters, theses and third-party funded projects for which the service was used need to be reported to us via [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] or via [mailto:sds-hd-support@urz.uni-heidelberg.de email].

== Registration Steps ==

The registration consists of two steps:

* '''Step A:''' Membership in Storage Project (Speichervorhaben/'''SV''', same as SDS@hd allocation). There are two options:
*: '''a)''' →[[#a)_Get_invited_to_existing_SV | Get invited to existing SV]]
*: '''b)''' →[[#b)_Apply_for_new_SV | Apply for new SV]]
* '''Step B:''' →[[#Step_B:_Registration_for_SDS@hd_Service |Registration for SDS@hd Service]]

After finishing the registration, the '''next steps''' are:
* Check which management options there are for your respective role in the SV → [[SDS@hd/SV_Management |SV Management]]
* See how to access and use your storage space → [[SDS@hd/Access | Access]]

[[File:Sds-hd-registration_new.png|center|x500px|upright|SDS@hd Registration Process]]

=== Step A: SV Membership ===
[[File:BwServices_Management_project_sv_structure.png|600px|thumb|right|A Project consists of multiple elements like a description, users and allocations/SVs. Every user in the project can be added to one or multiple of the available SVs.]]
You can either join an existing SV or start a new one. SVs are always part of a Project at the [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] page. In the image on the right you can see how the general setup of a Project looks like. The person who created the Project is called PI (Principal Investigator). The PI and the PMs (Project Managers), defined by the PI, can invite new users to the project and add them to SVs.

==== a) Get invited to existing SV ====

To join an existing SV as a coworker, ask the PI or a PM to...:
# ...add you to the project:
#* They can add you directly if you've already logged in to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] once. You get notified via email.
#* They can invite you by sending an email invite. You have to accept the invite to be a project member first, before you can be added to the SV.
# ...add you to the SV.
#*: You can see that you were added by going to the page of the bwServices-Project on [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] and then checking the list of allocations that you are a member of. You get notified via email as well.

→ As soon as you're a member of an SV, you can [[#Step_B:_Registration_for_SDS@hd_Service | go on with Step B]] if you are using SDS@hd for the first time.
If not, it's a good idea to trigger an update of your user data by logging in to bwServices once.

==== b) Apply for new SV ====

This is typically done only by the leader of a scientific work group or the senior scientist of a research group/collaboration.
You can add any amount of co-workers to your SV.

# '''Check prerequisites:''' Your institution must be a [https://www.bwidm.de/hochschulen.php bwIDM member] and you have to be an employee of your institution.
# '''Create a Project''' at [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] (alternatively use an already existing project):
## Go to the bwServices-Management Tool. At the start page click the title "Projects" and then "Add a project".
## Fill in and submit the form:
##* Keep in mind that this bwServices-Project can stand for a scientific research project but alternatively, it can for example represent all acitivities of a research group as well. The title and description can be changed later.
##* The information to fill in is as follows: <syntaxhighlight>
Please provide a detailed description of your work and select your field of science. In case that this Project corresponds to one dedicated scientific project, please describe your scientific project and indicate the expected project duration. Please mention additional infrastructure that is used by the project (cluster, cloud, storage, archiving services). Thank you!
</syntaxhighlight>
##* The provided address should be the one where a potential bill can be sent to. In cases where different SVs in the project would have different billing addresses, the address field in the SV application can be filled in additionally.
#: → Your project is directly active.
# '''Apply for SV'''. This is the same as adding allocations to a project:
## In the Allocations section of the Project's page is a button "Request Resource Allocation". Currently, the only allocation that can be chosen is an allocation for the resource SDS@hd. An SDS@hd allocation is called SV (Speichervorhaben / storage project).
## Fill in the form:
##* Justification questions (this field is the only one that can't be changed later on): <syntaxhighlight>
Q1: If not already provided in the bwServices-Management-Project description: Describe the intended usage of the storage space and indicate the expected duration that the allocation is needed.
Q2: How are data generated and processed? Please distinguish primary vs. secondary data and mention relevant equipment or software.
Q3: What is the expected data growth? Which file formats are used? Please describe any notable characteristics of your data, such as exceptionally large file size or very large number of small files.
Q4: Describe at which timepoints data can be archived or deleted. Describe the data and the data volume that can be archived at the end of the project.
Q5: How are your project data structured, and how is access managed? Are there special needs for the access rights within the SV (e.g. users or guest folder)?
</syntaxhighlight>
##* Title of the SV: The title is for you to recognize the SV in the allocations table if you have multiple SVs. It can be short.
## After filling in the form and agreeing to the terms, the allocation is shown on the project page with the status 'New'. A 'New' allocation is not approved, yet, and therefore not usable.
## The allocation application gets reviewed and approved. In the approval state no new users can be added. As soon as the storage space was set up, an email informs about the activation of the allocation.

→ [[#Step_B:_Registration_for_SDS@hd_Service | Go on with Step B]] if you are using SDS@hd for the first time.

If you're a PI in a project with SDS@hd allocations, you will be:
* ...held accountable for the co-workers in the SV
* ...responsible for all data stored in this SV, as well as for any costs incurred as a result.
* ...asked to provide information for the two reports required by the DFG for their funding of SDS@hd
* ...likely asked for a contribution to a future DFG grant proposal for an extension of the storage system in your area of research ("wissenschaftliches Beiblatt")

=== Step B: Registration for SDS@hd Service ===
[[File:bwServices_chooseHomeOrganisation.png|right|300px|thumb|Select your home organization]]

After step A you have to register your personal account on the storage system and set a service password.

* Visit the registration website '''bwServices''': [https://bwservices.uni-heidelberg.de/ https://bwservices.uni-heidelberg.de]
*# Select your home organization from the list and click ''Proceed''
*#: You will be directed to the ''Identity Provider'' of your home organisation
*# Enter your home-organisational user ID / username and your home-organisational password and click ''Login'' button
*#: You will be redirected back to bwServices
* '''Register''' for the SDS@hd service:
*# Under ''The following services are available'' select the service ''SDS@hd - Scientific Data Storage''.
*# Click ''Register''
* Set your personal '''service password''' for SDS@hd. This is used for accessing SVs.
* The registration is finished.
*: → [[#sds_registration_nextSteps|next steps]]

=== Change Service Password at bwServices ===

→ '''[[Registration/bwForCluster/Helix#Setting_a_New_Service_Password | Setting a New Service Password]]'''

== FAQ ==
* Is there a Test-SV to do technical tests?
*: Yes, there is a Test-SV for this purpose. You can write to the support to get invited. Please keep in mind that data gets regularly deleted and users may be removed after some time.
* I can login with different institutions/IDs (edugain, ORCID, university ID, university clinics ID,...). Which one shall I choose?
*: If you want to be able to start your own SVs, use the ID were you can get the entitlement most conveniently. Otherwise it doesn't matter. But ORCID is the most unspecific option and therefore should only be chosen if the other options don't work.
* I own a SV with my ID at institution A. Can I connect my account from institution B to it?
*: You can't connect different accounts. But you can join the SV with both accounts if necessary.
* What is my userID when I register via ORCID?
*: You'll be informed about the ID along the process. It'll start with "orcid_".
* How do I set up ORCID?
*: Login with your ORCID account (instead of an institutional account) and set it up so that your email address is shared with us.
* How do I register with foreign institutions (outside of [https://www.belwue.de/ueberuns/teilnehmer.html BelWü])
*: We might need to enable the IP range first.
*: After we've added the ip addresses, you might need to wait for an hour before it works.

== Troubleshooting ==

* '''Issue:''' The registration on bwServices is not possible
** Error message: You have no access to a Speichervorhaben of the Service SDS@hd. Please register a new Speichervorhaben or collaborate with an existing Project on https://management.bwservices.uni-heidelberg.de/
** Solution: Check the following points:
*** Are you a member of a bwServices-Project?
*** Does this project have an SDS@hd allocation?
*** Are you a member of this allocation/SV?
*** Is the bwServices-Project still active? The project mustn't be in the review process.
*** Is the allocation still active? The allocation mustn't be expired.

SDS@hd/Registration

2026-09-03T14:36:57Z

H Schumacher: Added troubleshooting section

== Prerequisites ==

* Only scientific data is stored. Don't place administrative data on SDS@hd.
* Only hot data is stored which means only data that is actively used is allowed. We expect you to remove data from finished projects. It is advised to use an archiving service for this. Temporary data should also be removed on a regular basis.
* The storage of personal data or data capable of being linked to an individual is not permitted.
* SDS@hd mustn't be used as an archive or a backup. To find solutions in this regard, please have a look at the [https://wiki.bwhpc.de/e/Main_Page Main Page] and your institutional services.
* The use of SDS@hd must be adequately documented:
** All publications that were created with the involvement of the storage service SDS@hd should acknowledge its use. For an example text see [[SDS@hd/Acknowledgement | here]].
** These publications, scientific posters, theses and third-party funded projects for which the service was used need to be reported to us via [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] or via [mailto:sds-hd-support@urz.uni-heidelberg.de email].

== Registration Steps ==

The registration consists of two steps:

* '''Step A:''' Membership in Storage Project (Speichervorhaben/'''SV''', same as SDS@hd allocation). There are two options:
*: '''a)''' →[[#a)_Get_invited_to_existing_SV | Get invited to existing SV]]
*: '''b)''' →[[#b)_Apply_for_new_SV | Apply for new SV]]
* '''Step B:''' →[[#Step_B:_Registration_for_SDS@hd_Service |Registration for SDS@hd Service]]

After finishing the registration, the '''next steps''' are:
* Check which management options there are for your respective role in the SV → [[SDS@hd/SV_Management |SV Management]]
* See how to access and use your storage space → [[SDS@hd/Access | Access]]

[[File:Sds-hd-registration_new.png|center|x500px|upright|SDS@hd Registration Process]]

=== Step A: SV Membership ===
[[File:BwServices_Management_project_sv_structure.png|600px|thumb|right|A Project consists of multiple elements like a description, users and allocations/SVs. Every user in the project can be added to one or multiple of the available SVs.]]
You can either join an existing SV or start a new one. SVs are always part of a Project at the [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] page. In the image on the right you can see how the general setup of a Project looks like. The person who created the Project is called PI (Principal Investigator). The PI and the PMs (Project Managers), defined by the PI, can invite new users to the project and add them to SVs.

==== a) Get invited to existing SV ====

To join an existing SV as a coworker, ask the PI or a PM to add you to the project and afterwards to the SV.
* They can add you directly if you've already logged in to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] once.
* They can invite you by sending an email invite. You have to accept the invite to be a project member first, before you can be added to the SV.
Whenever a manager adds you to a project or SV, you get an information via email.

→ As soon as you're a member of an SV, you can [[#Step_B:_Registration_for_SDS@hd_Service | go on with Step B]] if you are using SDS@hd for the first time.

==== b) Apply for new SV ====

This is typically done only by the leader of a scientific work group or the senior scientist of a research group/collaboration.
You can add any amount of co-workers to your SV.

# '''Check prerequisites:''' Your institution must be a [https://www.bwidm.de/hochschulen.php bwIDM member] and you have to be an employee of your institution.
# '''Create a Project''' at [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] (alternatively use an already existing project):
## Go to the bwServices-Management Tool. At the start page click the title "Projects" and then "Add a project".
## Fill in and submit the form:
##* Keep in mind that this bwServices-Project can stand for a scientific research project but alternatively, it can for example represent all acitivities of a research group as well. The title and description can be changed later.
##* The information to fill in is as follows: <syntaxhighlight>
Please provide a detailed description of your work and select your field of science. In case that this Project corresponds to one dedicated scientific project, please describe your scientific project and indicate the expected project duration. Please mention additional infrastructure that is used by the project (cluster, cloud, storage, archiving services). Thank you!
</syntaxhighlight>
##* The provided address should be the one where a potential bill can be sent to. In cases where different SVs in the project would have different billing addresses, the address field in the SV application can be filled in additionally.
#: → Your project is directly active.
# '''Apply for SV'''. This is the same as adding allocations to a project:
## In the Allocations section of the Project's page is a button "Request Resource Allocation". Currently, the only allocation that can be chosen is an allocation for the resource SDS@hd. An SDS@hd allocation is called SV (Speichervorhaben / storage project).
## Fill in the form:
##* Justification questions (this field is the only one that can't be changed later on): <syntaxhighlight>
Q1: If not already provided in the bwServices-Management-Project description: Describe the intended usage of the storage space and indicate the expected duration that the allocation is needed.
Q2: How are data generated and processed? Please distinguish primary vs. secondary data and mention relevant equipment or software.
Q3: What is the expected data growth? Which file formats are used? Please describe any notable characteristics of your data, such as exceptionally large file size or very large number of small files.
Q4: Describe at which timepoints data can be archived or deleted. Describe the data and the data volume that can be archived at the end of the project.
Q5: How are your project data structured, and how is access managed? Are there special needs for the access rights within the SV (e.g. users or guest folder)?
</syntaxhighlight>
##* Title of the SV: The title is for you to recognize the SV in the allocations table if you have multiple SVs. It can be short.
## After filling in the form and agreeing to the terms, the allocation is shown on the project page with the status 'New'. A 'New' allocation is not approved, yet, and therefore not usable.
## The allocation application gets reviewed and approved. In the approval state no new users can be added. As soon as the storage space was set up, an email informs about the activation of the allocation.

→ [[#Step_B:_Registration_for_SDS@hd_Service | Go on with Step B]] if you are using SDS@hd for the first time.

If you're a PI in a project with SDS@hd allocations, you will be:
* ...held accountable for the co-workers in the SV
* ...responsible for all data stored in this SV, as well as for any costs incurred as a result.
* ...asked to provide information for the two reports required by the DFG for their funding of SDS@hd
* ...likely asked for a contribution to a future DFG grant proposal for an extension of the storage system in your area of research ("wissenschaftliches Beiblatt")

=== Step B: Registration for SDS@hd Service ===
[[File:bwServices_chooseHomeOrganisation.png|right|300px|thumb|Select your home organization]]

After step A you have to register your personal account on the storage system and set a service password.

* Visit the registration website '''bwServices''': [https://bwservices.uni-heidelberg.de/ https://bwservices.uni-heidelberg.de]
*# Select your home organization from the list and click ''Proceed''
*#: You will be directed to the ''Identity Provider'' of your home organisation
*# Enter your home-organisational user ID / username and your home-organisational password and click ''Login'' button
*#: You will be redirected back to bwServices
* '''Register''' for the SDS@hd service:
*# Under ''The following services are available'' select the service ''SDS@hd - Scientific Data Storage''.
*# Click ''Register''
* Set your personal '''service password''' for SDS@hd. This is used for accessing SVs.
* The registration is finished.
*: → [[#sds_registration_nextSteps|next steps]]

=== Change Service Password at bwServices ===

→ '''[[Registration/bwForCluster/Helix#Setting_a_New_Service_Password | Setting a New Service Password]]'''

== FAQ ==
* Is there a Test-SV to do technical tests?
*: Yes, there is a Test-SV for this purpose. You can write to the support to get invited. Please keep in mind that data gets regularly deleted and users may be removed after some time.
* I can login with different institutions/IDs (edugain, ORCID, university ID, university clinics ID,...). Which one shall I choose?
*: If you want to be able to start your own SVs, use the ID were you can get the entitlement most conveniently. Otherwise it doesn't matter. But ORCID is the most unspecific option and therefore should only be chosen if the other options don't work.
* I own a SV with my ID at institution A. Can I connect my account from institution B to it?
*: You can't connect different accounts. But you can join the SV with both accounts if necessary.
* What is my userID when I register via ORCID?
*: You'll be informed about the ID along the process. It'll start with "orcid_".
* How do I set up ORCID?
*: Login with your ORCID account (instead of an institutional account) and set it up so that your email address is shared with us.
* How do I register with foreign institutions (outside of [https://www.belwue.de/ueberuns/teilnehmer.html BelWü])
*: We might need to enable the IP range first.
*: After we've added the ip addresses, you might need to wait for an hour before it works.

== Troubleshooting ==

* '''Issue:''' The registration on bwServices is not possible
** Error message: You have no access to a Speichervorhaben of the Service SDS@hd. Please register a new Speichervorhaben or collaborate with an existing Project on https://management.bwservices.uni-heidelberg.de/
** Solution: Check the following points:
*** Are you a member of a bwServices-Project?
*** Does this project have an SDS@hd allocation?
*** Are you a member of this allocation/SV?
*** Is the bwServices-Project still active? The project mustn't be in the review process.
*** Is the allocation still active? The allocation mustn't be expired.

SDS@hd/SV Management

2026-09-02T14:30:53Z

H Schumacher: added more information about chmod

{| style=" background:#FEF4AB; width:100%;"
| style="padding:8px; background:#FFE856; font-size:120%; text-align:left" | For (previous) users of the SDS@hd Management Tool:
|-
|
On the 1st of September we're going to replace the SDS@hd Management website with the [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] website. 
The '''main changes in the registration and management process''' are as follows:
* An SV must be a part of a bwServices-Project.
* An SV is an ''Allocation'' requested from the ''Resource'' SDS@hd.
* The SVV role was removed. SVVs are now the PI (Principal Investigator) of a bwServices-Project that contains the SV.
* The SV-Manager role was removed. Managers (PMs) can be assigned within the project and have mostly the same rights as the PI.
* Publications can be added to the project at any time.
* New SV members can directly be added by one of the PMs. There is no need for sharing an SV-password and filling in a collaboration form anymore.
* New SV members are always added as 'Users' (= full members). If you want to add guests, please directly change their role to 'Guest' after adding them!

'''Important points for the migration:'''
The data of current SVs were migrated to the new page:
* The SVV is now the PI of a project and the project has an SDS@hd allocation.
* Inactive members of the SVs were removed.
* <ins>Initial tasks for the PIs</ins>:
*: Login to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] and update the following points:
** In the project:
*** Check and update the data (project description and attributes).
*** Define managers. Read their role description first! Check their rights in the [[#Project_Manager_(PM) | context of the project]] and in [[#PMs | context of the allocation]] (SV).
** In the allocation/SV:
*** Check the members list and if applicable the guest roles.
*** Check the attributes of the allocation (access options, quota, ...). They can be changed via [[#Adjust_SV_Attributes | Change Request]].
*** Billing data must only be provided if the SV generates costs according to the [https://sds-hd.urz.uni-heidelberg.de/management/shib/en/sds_costs_download.php price sheet] and if additionally, the billing information is different from the information provided in the project.
* The SDS@hd Management page will stay in read-only mode for a short while before it gets deactivated.
* Test-SV: Members that weren't active in the last 30 days were removed. Access to the Test-SV is given on request to the SDS@hd Support.
* Inactive PI: We'll get in contact with projects that don't have an active PI.
|}
This page provides an overview of options to structure projects that use SDS@hd. The simple use case is:
* One research project has one bwServices-Management Project (in short bwServices-Project).
* One bwServices-Project has one SV.
* Both have the same group of members.
The graphic below visualizes one option for another setup. Depending on your preferences of who should have access to which data, you can structure things differently.
[[File:BwServices_Management_project_sv_structure.png|600px|thumb|right|A Project consists of multiple elements like a description, users and allocations/SVs. Every user in the project can be added to one or multiple of the available SVs.]]

= bwServices-Management =

In bwServices-Management the top level entity is a project. On this page ''project'' always refers to this ''bwServices-Project''. Every bwIDM member with employee status can create a project by going to the ''Projects'' page (Home -> Projects) and then clicking on the "Add a project" button. A project can have multiple users. For specifics regarding the different roles in a project see the [[#Roles_in_a_Project| Roles]] section.
[[File:bwServices_Management_home.png|300px|thumb|right|The start page lists recent projects and allocations. The titles link to the complete lists.]]

'''Adding users to a project:'''
# On the project's page either the button on the top or next to the users list can be used for adding users.
# There are two ways to add new project members:
#:* Option 1: To invite someone that already logged in to the bwServices-Management page once, they can be found...
#:** ... via their user id. A list of ids can be provided in the search field.
#:** ... by providing (a part of) the name and searching "All Fields".
#:* Option 2: Alternatively, an invite link can be sent by providing the email address. The email address for sending the invite doesn't need to be the address that is later on used for their account on bwServices-Management. When a user clicks on the invite link they have to login to bwServices-Management. Afterwards, they are a part of the project with the account that they logged in with.
# Before adding the person or submitting the invite, the role of the new project member should be adjusted if needed. Available roles are "[[#User |User]]" or "[[#Project_Manager_(PM) |Manager]]".
# The added person gets an email notification regarding their new project membership.

A project can have multiple allocations. Having an allocation means that you can allocate a resource like SDS@hd for your project. An allocation comes with its own form to fill in and its own terms to agree to. You can find detailed instructions under [[SDS@hd/Registration#b)_Apply_for_new_SV | Registration/Apply for new SV]].

'''Adding users to an allocation:'''

An allocation can have multiple users. The users can be chosen from the list of project members.

Here is an overview of the most important pages and what you can find there:
[[File:bwServices_Management_pagetree.png|600px|center|bwServices-Management Page Overview]]

== Roles in a Project ==
There are three different roles in a project. Depending on the role, different management activites can be executed at [https://management.bwservices.uni-heidelberg.de/ bwServices-Management]. These are described below. They include top level management of user access.

=== User ===
Users can...
* ...view project information.
* ...view allocations that they are a member of.

* ...enable/disable notifications. Notifications regarding the Projects that they are (or will be) a member of can be:
** information that they were added to the project
** emails from one of the project's PMs
** information about a project being archived

=== Project Manager (PM) ===
Additionally to the abilities of a User, a PM can...
* ...add and remove Users from the project.
* ...add and remove Users from allocations.
* ...change the roles of Users on project and allocation level (Project: User <-> PM; SV: None <-> Guest).

* ...enable/disable notifications for other Users.
* ...always receive email notifications.
* ...write an email to all Project Users.

* ...request new allocations.
* ...change project attributes.
* ...add publications.
* ...extend project lifetime.
* ...extend allocation lifetime.
* ...can archive (~ delete) the project.

==== Yearly Project Extension ====

Once a year a PM in the project needs to apply for a lifetime extension. This is indicated by the label "Project Review". Please update your project information first before applying for the extension. Especially, update the list of publications for which the allocated resources (e.g. SDS@hd) were used.

==== Archiving a Project ====
Caution: Same as '''deleting''' all allocations!

For PMs there is a button for archiving the project in the upper right part of the project page. Archiving means that the status of the allocations changes to "expired" and the [[#SV_Deprovisioning | deprovisioning]] process starts. The project with its description will be deprovisioned some time after all allocations are removed.

=== Principal Investigator (PI) ===
A PI is automatically a PM as well. -> Mentions of 'PM' always include the PI.
Additionally to the abilities of a PM, the PI...
* ...is responsible for the whole project. This includes all allocations, independently of who requested them.
* ...is automatically a member of every allocation.

== Roles in a SV ==

The SV users keep their project roles (PI/PM/User). Additionally, they can hold the Guest role in an SV. The roles within the context of an SV are described below.

=== All SV Users ===
At [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] they can...
* ...view their role in the SV. If no Role column is shown, there are no users with a Guest role in the SV.
* ...view the status of the SV (active/expired, end date)
Notifications regarding the SVs that they are a member of can be:
** information about having been added to an allocation
** information about accepted or denied change requests (without the concrete content of the request)
** information about an allocation being activated or denied

=== Guests ===
PMs can add the "Guest" role to a user in the SV by going to the users list of the allocation and clicking the pencil button in the last column of the user's row. This leads to the details of the user where the role can be changed.
As soon as the first user is defined as Guest via bwServices-Management, a "guest" folder is created in the SV. A Guest can see but not access content in the SV root. They can only work within the "guest" folder. For more information see [[#Guest_Folder | Guest Folder]].
'''Caution:''' When adding new guests to the Project and afterwards to the SV, make sure to directly change their role in the SV from the default to "Guest". Otherwise they have full access first.

=== PMs ===
PMs in the project can...
* ...manage the SV as mentioned in their role description.
* ...adjust SV attributes via change request. Previous change requests can be viewed.
* ...extend the SV lifetime.
* ...request the deletion of the SV.

==== Adjust SV Attributes ====

To change the properties of your SV, you can hand in a Change Request.
When you request a quota expansion for your SV at the SDS@hd service in order to store hot scientific data, we kindly ask you to answer the questions in the justification textfield for reasoning the extension. The questions are:
<syntaxhighlight>Q1: Please explain why additional storage is needed.
Q2: What is the potential for deleting or archiving data?</syntaxhighlight>

Please also consider the following remarks:
Upon increasing storage capacities, there will also be a need for further activities in the reporting from your side, e.g. by pointing out the usage of SDS@hd in scientific publications to us. More information about this can be found here: https://wiki.bwhpc.de/e/SDS@hd/Acknowledgement
Please consider raising funds for the usage and further operation of the SDS@hd service. We will gladly support you with this.

==== Yearly SV Extension ====

Once a year a PM needs to apply for a lifetime extension of the storage project. This is indicated by the label "Allocation Renewal". As long as SDS@hd is used appropriately, an extension will be granted.
Warning: You have to apply for the extension in the given timeframe. When you're too late, you have to contact the sds-support to get the extension. SVs that don't get extended, are going to be deleted by us, see [[#SV_Deprovisioning | SV Deprovisioning]]. Before the deletion, the PI is informed via email.

==== SV Deprovisioning ====
The deprovisioning is initiated in the following cases:
* The SV is not extended.
* The Project was archived.
* A PM that is a member of the SV asks via Change Request for a deletion of the SV by writing this request in the "Justification for Changes" field.

The deprovisioning basically contains the following steps:
* The access of the SV-group to the SV is blocked. The access to any data that is saved within the SV is not possible anymore.
* After a specific period of time all data in the SV gets permanently deleted.

== FAQ ==

* I changed a user's role in bwServices-Management but the user still has the previous access rights.
*: After 30 minutes the adjustment should be synced to all systems. If not, it helps when the member with the role change logs in to bwServices once to trigger an automatic update and then waits for 30 minutes.
* How can I remove myself from an allocation or project?
*: This is not possible. You must ask a PM to remove you.
* How can I deactivate Users?
*: Users can't be deactivated. They can only be removed from allocations or projects.
* My project or my allocation ist expired. How can I prolong it?
*: Please [mailto:sds-hd-support@urz.uni-heidelberg.de submit a Ticket]
* How can the PI of a project be changed?
*: This is currently not possible but will be possible in the future.

= Within the SV =
While bwServices-Management manages the access rights and storage amount on a top level outside of the storage space itself, there are multiple options to organize data and adjust access rights when being connected to the SV.

== Default Access Rights ==

* In the main space of an SV everyone has read and write access.
* The person who creates a file or folder is the owner. The owner can adjust the rights via [[Workspace#Regular_Unix_Permissions | chmod]] when using one of the following connection methods: sftp/sshfs, nfs, bwVisu, bwForCluster Helix. On Windows the rights can be set with a right click on the file/folder and using the properties menu.
* The SV responsible has full control over the access rights by being able to take [[#Taking_Ownership_of_Files | ownership of files]].
* The SDS@hd support can help with defining access rules for your specific use case. Optimally, this is discussed before the SV gets filled with data. Examples:
** A group folder with read only access in contrast to the default read and write access. The SV members can see everything from each other but not change files that aren't their own.
** A users folder where every user can create their own folder with content that is inaccessible to others.

As an example, let's assume that the SV is structured as shown in the graphic below.
[[File:SDS_AccessRules_Overview.jpg|600px|center|Default access rules in a SV]]

In the SV scheme on the left you can see that the SV has two guests, two full members and the PI. The full members have read and write access to all normal folders in the SV root and the `guest` folder.

Additionally, there can be a `users` folder with a special setup. In the `users` folder everyone has read access but only the owners of the folders/files have write access.
<ins>Hint:</ins> If needed, the SDS@hd support can create custom access rules for specific top level folders.

The SV scheme on the right visualizes how the orange full member experiences the access when working in the SV. The full member has free read and write access to everything except the personal folder of the blue user.

== Guest Folder ==
Using the option to add some SV members with the role "Guest" allows better access management for separate user groups that shall not get access to the whole SV.

'''Use Cases:'''
* Data exchange with core facilities.
* Projects together with external collaborators.
* Up- and Download area

'''Setup:'''
* PMs can enable the guest folder by adding guests to the SV see [[#Guests | Guests]]. The folder gets created as soon as the first Guest user is defined. The creation might need up to 20 minutes.

'''Access Rights'''
* Full SV member:
** Read and write access to all content of the SV including the guest folder.
** No access to content that was deliberately made unaccessible by other SV members via <code>chmod</code>.
** No access to 'user folders'.
* Guest:
** Folder above guest folder:
*** Can see all file and folder names. Can't see content in subfolders.
*** Can't access or create content.
** Guest folder:
*** Can only create folders, no files, in the root of the guest folder.
*** Read content. Read content in subfolders.
**** Alternative setup: Read content. Can't read content in subfolders / folders from other guests. Please contact the sds-support to get this setup.
*** Has only write access to content that they created themselves.

The following graphic shows the current default access rights in the guest folder on the left in contrast to the alternative setup on the right.
[[File:SDS_AccessRules_Guest.jpg|600px|center|Guest folder default vs. custom]]

== Storage Space Management ==

At the moment you can't see how much storage space is left. We are working on including this information.

To keep the data hot, you should regularly check the following points:
* Make sure that temporary data gets removed.
* Make sure that no data from finished projects is stored. You might want to archive those with one of the archiving services listed on the [https://wiki.bwhpc.de/e/Main_Page Main Page].
* If someone has left the storage project, make sure that the PI has access to the data and the data is still hot.

SDS@hd/SV Management

2026-09-02T14:25:56Z

H Schumacher: fixed link

{| style=" background:#FEF4AB; width:100%;"
| style="padding:8px; background:#FFE856; font-size:120%; text-align:left" | For (previous) users of the SDS@hd Management Tool:
|-
|
On the 1st of September we're going to replace the SDS@hd Management website with the [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] website. 
The '''main changes in the registration and management process''' are as follows:
* An SV must be a part of a bwServices-Project.
* An SV is an ''Allocation'' requested from the ''Resource'' SDS@hd.
* The SVV role was removed. SVVs are now the PI (Principal Investigator) of a bwServices-Project that contains the SV.
* The SV-Manager role was removed. Managers (PMs) can be assigned within the project and have mostly the same rights as the PI.
* Publications can be added to the project at any time.
* New SV members can directly be added by one of the PMs. There is no need for sharing an SV-password and filling in a collaboration form anymore.
* New SV members are always added as 'Users' (= full members). If you want to add guests, please directly change their role to 'Guest' after adding them!

'''Important points for the migration:'''
The data of current SVs were migrated to the new page:
* The SVV is now the PI of a project and the project has an SDS@hd allocation.
* Inactive members of the SVs were removed.
* <ins>Initial tasks for the PIs</ins>:
*: Login to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] and update the following points:
** In the project:
*** Check and update the data (project description and attributes).
*** Define managers. Read their role description first! Check their rights in the [[#Project_Manager_(PM) | context of the project]] and in [[#PMs | context of the allocation]] (SV).
** In the allocation/SV:
*** Check the members list and if applicable the guest roles.
*** Check the attributes of the allocation (access options, quota, ...). They can be changed via [[#Adjust_SV_Attributes | Change Request]].
*** Billing data must only be provided if the SV generates costs according to the [https://sds-hd.urz.uni-heidelberg.de/management/shib/en/sds_costs_download.php price sheet] and if additionally, the billing information is different from the information provided in the project.
* The SDS@hd Management page will stay in read-only mode for a short while before it gets deactivated.
* Test-SV: Members that weren't active in the last 30 days were removed. Access to the Test-SV is given on request to the SDS@hd Support.
* Inactive PI: We'll get in contact with projects that don't have an active PI.
|}
This page provides an overview of options to structure projects that use SDS@hd. The simple use case is:
* One research project has one bwServices-Management Project (in short bwServices-Project).
* One bwServices-Project has one SV.
* Both have the same group of members.
The graphic below visualizes one option for another setup. Depending on your preferences of who should have access to which data, you can structure things differently.
[[File:BwServices_Management_project_sv_structure.png|600px|thumb|right|A Project consists of multiple elements like a description, users and allocations/SVs. Every user in the project can be added to one or multiple of the available SVs.]]

= bwServices-Management =

In bwServices-Management the top level entity is a project. On this page ''project'' always refers to this ''bwServices-Project''. Every bwIDM member with employee status can create a project by going to the ''Projects'' page (Home -> Projects) and then clicking on the "Add a project" button. A project can have multiple users. For specifics regarding the different roles in a project see the [[#Roles_in_a_Project| Roles]] section.
[[File:bwServices_Management_home.png|300px|thumb|right|The start page lists recent projects and allocations. The titles link to the complete lists.]]

'''Adding users to a project:'''
# On the project's page either the button on the top or next to the users list can be used for adding users.
# There are two ways to add new project members:
#:* Option 1: To invite someone that already logged in to the bwServices-Management page once, they can be found...
#:** ... via their user id. A list of ids can be provided in the search field.
#:** ... by providing (a part of) the name and searching "All Fields".
#:* Option 2: Alternatively, an invite link can be sent by providing the email address. The email address for sending the invite doesn't need to be the address that is later on used for their account on bwServices-Management. When a user clicks on the invite link they have to login to bwServices-Management. Afterwards, they are a part of the project with the account that they logged in with.
# Before adding the person or submitting the invite, the role of the new project member should be adjusted if needed. Available roles are "[[#User |User]]" or "[[#Project_Manager_(PM) |Manager]]".
# The added person gets an email notification regarding their new project membership.

A project can have multiple allocations. Having an allocation means that you can allocate a resource like SDS@hd for your project. An allocation comes with its own form to fill in and its own terms to agree to. You can find detailed instructions under [[SDS@hd/Registration#b)_Apply_for_new_SV | Registration/Apply for new SV]].

'''Adding users to an allocation:'''

An allocation can have multiple users. The users can be chosen from the list of project members.

Here is an overview of the most important pages and what you can find there:
[[File:bwServices_Management_pagetree.png|600px|center|bwServices-Management Page Overview]]

== Roles in a Project ==
There are three different roles in a project. Depending on the role, different management activites can be executed at [https://management.bwservices.uni-heidelberg.de/ bwServices-Management]. These are described below. They include top level management of user access.

=== User ===
Users can...
* ...view project information.
* ...view allocations that they are a member of.

* ...enable/disable notifications. Notifications regarding the Projects that they are (or will be) a member of can be:
** information that they were added to the project
** emails from one of the project's PMs
** information about a project being archived

=== Project Manager (PM) ===
Additionally to the abilities of a User, a PM can...
* ...add and remove Users from the project.
* ...add and remove Users from allocations.
* ...change the roles of Users on project and allocation level (Project: User <-> PM; SV: None <-> Guest).

* ...enable/disable notifications for other Users.
* ...always receive email notifications.
* ...write an email to all Project Users.

* ...request new allocations.
* ...change project attributes.
* ...add publications.
* ...extend project lifetime.
* ...extend allocation lifetime.
* ...can archive (~ delete) the project.

==== Yearly Project Extension ====

Once a year a PM in the project needs to apply for a lifetime extension. This is indicated by the label "Project Review". Please update your project information first before applying for the extension. Especially, update the list of publications for which the allocated resources (e.g. SDS@hd) were used.

==== Archiving a Project ====
Caution: Same as '''deleting''' all allocations!

For PMs there is a button for archiving the project in the upper right part of the project page. Archiving means that the status of the allocations changes to "expired" and the [[#SV_Deprovisioning | deprovisioning]] process starts. The project with its description will be deprovisioned some time after all allocations are removed.

=== Principal Investigator (PI) ===
A PI is automatically a PM as well. -> Mentions of 'PM' always include the PI.
Additionally to the abilities of a PM, the PI...
* ...is responsible for the whole project. This includes all allocations, independently of who requested them.
* ...is automatically a member of every allocation.

== Roles in a SV ==

The SV users keep their project roles (PI/PM/User). Additionally, they can hold the Guest role in an SV. The roles within the context of an SV are described below.

=== All SV Users ===
At [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] they can...
* ...view their role in the SV. If no Role column is shown, there are no users with a Guest role in the SV.
* ...view the status of the SV (active/expired, end date)
Notifications regarding the SVs that they are a member of can be:
** information about having been added to an allocation
** information about accepted or denied change requests (without the concrete content of the request)
** information about an allocation being activated or denied

=== Guests ===
PMs can add the "Guest" role to a user in the SV by going to the users list of the allocation and clicking the pencil button in the last column of the user's row. This leads to the details of the user where the role can be changed.
As soon as the first user is defined as Guest via bwServices-Management, a "guest" folder is created in the SV. A Guest can see but not access content in the SV root. They can only work within the "guest" folder. For more information see [[#Guest_Folder | Guest Folder]].
'''Caution:''' When adding new guests to the Project and afterwards to the SV, make sure to directly change their role in the SV from the default to "Guest". Otherwise they have full access first.

=== PMs ===
PMs in the project can...
* ...manage the SV as mentioned in their role description.
* ...adjust SV attributes via change request. Previous change requests can be viewed.
* ...extend the SV lifetime.
* ...request the deletion of the SV.

==== Adjust SV Attributes ====

To change the properties of your SV, you can hand in a Change Request.
When you request a quota expansion for your SV at the SDS@hd service in order to store hot scientific data, we kindly ask you to answer the questions in the justification textfield for reasoning the extension. The questions are:
<syntaxhighlight>Q1: Please explain why additional storage is needed.
Q2: What is the potential for deleting or archiving data?</syntaxhighlight>

Please also consider the following remarks:
Upon increasing storage capacities, there will also be a need for further activities in the reporting from your side, e.g. by pointing out the usage of SDS@hd in scientific publications to us. More information about this can be found here: https://wiki.bwhpc.de/e/SDS@hd/Acknowledgement
Please consider raising funds for the usage and further operation of the SDS@hd service. We will gladly support you with this.

==== Yearly SV Extension ====

Once a year a PM needs to apply for a lifetime extension of the storage project. This is indicated by the label "Allocation Renewal". As long as SDS@hd is used appropriately, an extension will be granted.
Warning: You have to apply for the extension in the given timeframe. When you're too late, you have to contact the sds-support to get the extension. SVs that don't get extended, are going to be deleted by us, see [[#SV_Deprovisioning | SV Deprovisioning]]. Before the deletion, the PI is informed via email.

==== SV Deprovisioning ====
The deprovisioning is initiated in the following cases:
* The SV is not extended.
* The Project was archived.
* A PM that is a member of the SV asks via Change Request for a deletion of the SV by writing this request in the "Justification for Changes" field.

The deprovisioning basically contains the following steps:
* The access of the SV-group to the SV is blocked. The access to any data that is saved within the SV is not possible anymore.
* After a specific period of time all data in the SV gets permanently deleted.

== FAQ ==

* I changed a user's role in bwServices-Management but the user still has the previous access rights.
*: After 30 minutes the adjustment should be synced to all systems. If not, it helps when the member with the role change logs in to bwServices once to trigger an automatic update and then waits for 30 minutes.
* How can I remove myself from an allocation or project?
*: This is not possible. You must ask a PM to remove you.
* How can I deactivate Users?
*: Users can't be deactivated. They can only be removed from allocations or projects.
* My project or my allocation ist expired. How can I prolong it?
*: Please [mailto:sds-hd-support@urz.uni-heidelberg.de submit a Ticket]
* How can the PI of a project be changed?
*: This is currently not possible but will be possible in the future.

= Within the SV =
While bwServices-Management manages the access rights and storage amount on a top level outside of the storage space itself, there are multiple options to organize data and adjust access rights when being connected to the SV.

== Default Access Rights ==

* In the main space of an SV everyone has read and write access.
* The person who creates a file or folder is the owner. The owner can adjust the rights via [[Workspace#Regular_Unix_Permissions | chmod]].
* The SV responsible has full control over the access rights by being able to take [[#Taking_Ownership_of_Files | ownership of files]].
* The SDS@hd support can help with defining access rules for your specific use case. Optimally, this is discussed before the SV gets filled with data. Examples:
** A group folder with read only access in contrast to the default read and write access. The SV members can see everything from each other but not change files that aren't their own.
** A users folder where every user can create their own folder with content that is inaccessible to others.

As an example, let's assume that the SV is structured as shown in the graphic below.
[[File:SDS_AccessRules_Overview.jpg|600px|center|Default access rules in a SV]]

In the SV scheme on the left you can see that the SV has two guests, two full members and the PI. The full members have read and write access to all normal folders in the SV root and the `guest` folder.

Additionally, there can be a `users` folder with a special setup. In the `users` folder everyone has read access but only the owners of the folders/files have write access.
<ins>Hint:</ins> If needed, the SDS@hd support can create custom access rules for specific top level folders.

The SV scheme on the right visualizes how the orange full member experiences the access when working in the SV. The full member has free read and write access to everything except the personal folder of the blue user.

== Guest Folder ==
Using the option to add some SV members with the role "Guest" allows better access management for separate user groups that shall not get access to the whole SV.

'''Use Cases:'''
* Data exchange with core facilities.
* Projects together with external collaborators.
* Up- and Download area

'''Setup:'''
* PMs can enable the guest folder by adding guests to the SV see [[#Guests | Guests]]. The folder gets created as soon as the first Guest user is defined. The creation might need up to 20 minutes.

'''Access Rights'''
* Full SV member:
** Read and write access to all content of the SV including the guest folder.
** No access to content that was deliberately made unaccessible by other SV members via <code>chmod</code>.
** No access to 'user folders'.
* Guest:
** Folder above guest folder:
*** Can see all file and folder names. Can't see content in subfolders.
*** Can't access or create content.
** Guest folder:
*** Can only create folders, no files, in the root of the guest folder.
*** Read content. Read content in subfolders.
**** Alternative setup: Read content. Can't read content in subfolders / folders from other guests. Please contact the sds-support to get this setup.
*** Has only write access to content that they created themselves.

The following graphic shows the current default access rights in the guest folder on the left in contrast to the alternative setup on the right.
[[File:SDS_AccessRules_Guest.jpg|600px|center|Guest folder default vs. custom]]

== Storage Space Management ==

At the moment you can't see how much storage space is left. We are working on including this information.

To keep the data hot, you should regularly check the following points:
* Make sure that temporary data gets removed.
* Make sure that no data from finished projects is stored. You might want to archive those with one of the archiving services listed on the [https://wiki.bwhpc.de/e/Main_Page Main Page].
* If someone has left the storage project, make sure that the PI has access to the data and the data is still hot.

SDS@hd/SV Management

2026-09-02T10:39:53Z

H Schumacher: fixed link

{| style=" background:#FEF4AB; width:100%;"
| style="padding:8px; background:#FFE856; font-size:120%; text-align:left" | For (previous) users of the SDS@hd Management Tool:
|-
|
On the 1st of September we're going to replace the SDS@hd Management website with the [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] website. 
The '''main changes in the registration and management process''' are as follows:
* An SV must be a part of a bwServices-Project.
* An SV is an ''Allocation'' requested from the ''Resource'' SDS@hd.
* The SVV role was removed. SVVs are now the PI (Principal Investigator) of a bwServices-Project that contains the SV.
* The SV-Manager role was removed. Managers (PMs) can be assigned within the project and have mostly the same rights as the PI.
* Publications can be added to the project at any time.
* New SV members can directly be added by one of the PMs. There is no need for sharing an SV-password and filling in a collaboration form anymore.
* New SV members are always added as 'Users' (= full members). If you want to add guests, please directly change their role to 'Guest' after adding them!

'''Important points for the migration:'''
The data of current SVs were migrated to the new page:
* The SVV is now the PI of a project and the project has an SDS@hd allocation.
* Inactive members of the SVs were removed.
* <ins>Initial tasks for the PIs</ins>:
*: Login to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] and update the following points:
** In the project:
*** Check and update the data (project description and attributes).
*** Define managers. Read their role description first! Check their rights in the [[#Project_Manager_(PM) | context of the project]] and in [[#PMs | context of the allocation]] (SV).
** In the allocation/SV:
*** Check the members list and if applicable the guest roles.
*** Check the attributes of the allocation (access options, quota, ...). They can be changed via [[#Adjust_SV_Attributes | Change Request]].
*** Billing data must only be provided if the SV generates costs according to the [https://sds-hd.urz.uni-heidelberg.de/management/shib/en/sds_costs_download.php price sheet] and if additionally, the billing information is different from the information provided in the project.
* The SDS@hd Management page will stay in read-only mode for a short while before it gets deactivated.
* Test-SV: Members that weren't active in the last 30 days were removed. Access to the Test-SV is given on request to the SDS@hd Support.
* Inactive PI: We'll get in contact with projects that don't have an active PI.
|}
This page provides an overview of options to structure projects that use SDS@hd. The simple use case is:
* One research project has one bwServices-Management Project (in short bwServices-Project).
* One bwServices-Project has one SV.
* Both have the same group of members.
The graphic below visualizes one option for another setup. Depending on your preferences of who should have access to which data, you can structure things differently.
[[File:BwServices_Management_project_sv_structure.png|600px|thumb|right|A Project consists of multiple elements like a description, users and allocations/SVs. Every user in the project can be added to one or multiple of the available SVs.]]

= bwServices-Management =

In bwServices-Management the top level entity is a project. On this page ''project'' always refers to this ''bwServices-Project''. Every bwIDM member with employee status can create a project by going to the ''Projects'' page (Home -> Projects) and then clicking on the "Add a project" button. A project can have multiple users. For specifics regarding the different roles in a project see the [[#Roles_in_a_Project| Roles]] section.
[[File:bwServices_Management_home.png|300px|thumb|right|The start page lists recent projects and allocations. The titles link to the complete lists.]]

'''Adding users to a project:'''
# On the project's page either the button on the top or next to the users list can be used for adding users.
# There are two ways to add new project members:
#:* Option 1: To invite someone that already logged in to the bwServices-Management page once, they can be found...
#:** ... via their user id. A list of ids can be provided in the search field.
#:** ... by providing (a part of) the name and searching "All Fields".
#:* Option 2: Alternatively, an invite link can be sent by providing the email address. The email address for sending the invite doesn't need to be the address that is later on used for their account on bwServices-Management. When a user clicks on the invite link they have to login to bwServices-Management. Afterwards, they are a part of the project with the account that they logged in with.
# Before adding the person or submitting the invite, the role of the new project member should be adjusted if needed. Available roles are "[[#User |User]]" or "[[#Project_Manager_(PM) |Manager]]".
# The added person gets an email notification regarding their new project membership.

A project can have multiple allocations. Having an allocation means that you can allocate a resource like SDS@hd for your project. An allocation comes with its own form to fill in and its own terms to agree to. You can find detailed instructions under [[SDS@hd/Registration#b)_Apply_for_new_SV | Registration/Apply for new SV]].

'''Adding users to an allocation:'''

An allocation can have multiple users. The users can be chosen from the list of project members.

Here is an overview of the most important pages and what you can find there:
[[File:bwServices_Management_pagetree.png|600px|center|bwServices-Management Page Overview]]

== Roles in a Project ==
There are three different roles in a project. Depending on the role, different management activites can be executed at [https://management.bwservices.uni-heidelberg.de/ bwServices-Management]. These are described below. They include top level management of user access.

=== User ===
Users can...
* ...view project information.
* ...view allocations that they are a member of.

* ...enable/disable notifications. Notifications regarding the Projects that they are (or will be) a member of can be:
** information that they were added to the project
** emails from one of the project's PMs
** information about a project being archived

=== Project Manager (PM) ===
Additionally to the abilities of a User, a PM can...
* ...add and remove Users from the project.
* ...add and remove Users from allocations.
* ...change the roles of Users on project and allocation level (Project: User <-> PM; SV: None <-> Guest).

* ...enable/disable notifications for other Users.
* ...always receive email notifications.
* ...write an email to all Project Users.

* ...request new allocations.
* ...change project attributes.
* ...add publications.
* ...extend project lifetime.
* ...extend allocation lifetime.
* ...can archive (~ delete) the project.

==== Yearly Project Extension ====

Once a year a PM in the project needs to apply for a lifetime extension. This is indicated by the label "Project Review". Please update your project information first before applying for the extension. Especially, update the list of publications for which the allocated resources (e.g. SDS@hd) were used.

==== Archiving a Project ====
Caution: Same as '''deleting''' all allocations!

For PMs there is a button for archiving the project in the upper right part of the project page. Archiving means that the status of the allocations changes to "expired" and the [[#SV_Deprovisioning | deprovisioning]] process starts. The project with its description will be deprovisioned some time after all allocations are removed.

=== Principal Investigator (PI) ===
A PI is automatically a PM as well. -> Mentions of 'PM' always include the PI.
Additionally to the abilities of a PM, the PI...
* ...is responsible for the whole project. This includes all allocations, independently of who requested them.
* ...is automatically a member of every allocation.

== Roles in a SV ==

The SV users keep their project roles (PI/PM/User). Additionally, they can hold the Guest role in an SV. The roles within the context of an SV are described below.

=== All SV Users ===
At [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] they can...
* ...view their role in the SV. If no Role column is shown, there are no users with a Guest role in the SV.
* ...view the status of the SV (active/expired, end date)
Notifications regarding the SVs that they are a member of can be:
** information about having been added to an allocation
** information about accepted or denied change requests (without the concrete content of the request)
** information about an allocation being activated or denied

=== Guests ===
PMs can add the "Guest" role to a user in the SV by going to the users list of the allocation and clicking the pencil button in the last column of the user's row. This leads to the details of the user where the role can be changed.
As soon as the first user is defined as Guest via bwServices-Management, a "guest" folder is created in the SV. A Guest can see but not access content in the SV root. They can only work within the "guest" folder. For more information see [[#Guest_Folder | Guest Folder]].
'''Caution:''' When adding new guests to the Project and afterwards to the SV, make sure to directly change their role in the SV from the default to "Guest". Otherwise they have full access first.

=== PMs ===
PMs in the project can...
* ...manage the SV as mentioned in their role description.
* ...adjust SV attributes via change request. Previous change requests can be viewed.
* ...extend the SV lifetime.
* ...request the deletion of the SV.

==== Adjust SV Attributes ====

To change the properties of your SV, you can hand in a Change Request.
When you request a quota expansion for your SV at the SDS@hd service in order to store hot scientific data, we kindly ask you to answer the questions in the justification textfield for reasoning the extension. The questions are:
<syntaxhighlight>Q1: Please explain why additional storage is needed.
Q2: What is the potential for deleting or archiving data?</syntaxhighlight>

Please also consider the following remarks:
Upon increasing storage capacities, there will also be a need for further activities in the reporting from your side, e.g. by pointing out the usage of SDS@hd in scientific publications to us. More information about this can be found here: https://wiki.bwhpc.de/e/SDS@hd/Acknowledgement
Please consider raising funds for the usage and further operation of the SDS@hd service. We will gladly support you with this.

==== Yearly SV Extension ====

Once a year a PM needs to apply for a lifetime extension of the storage project. This is indicated by the label "Allocation Renewal". As long as SDS@hd is used appropriately, an extension will be granted.
Warning: You have to apply for the extension in the given timeframe. When you're too late, you have to contact the sds-support to get the extension. SVs that don't get extended, are going to be deleted by us, see [[#SV_Deprovisioning | SV Deprovisioning]]. Before the deletion, the PI is informed via email.

==== SV Deprovisioning ====
The deprovisioning is initiated in the following cases:
* The SV is not extended.
* The Project was archived.
* A PM that is a member of the SV asks via Change Request for a deletion of the SV by writing this request in the "Justification for Changes" field.

The deprovisioning basically contains the following steps:
* The access of the SV-group to the SV is blocked. The access to any data that is saved within the SV is not possible anymore.
* After a specific period of time all data in the SV gets permanently deleted.

== FAQ ==

* I changed a user's role in bwServices-Management but the user still has the previous access rights.
*: After 30 minutes the adjustment should be synced to all systems. If not, it helps when the member with the role change logs in to bwServices once to trigger an automatic update and then waits for 30 minutes.
* How can I remove myself from an allocation or project?
*: This is not possible. You must ask a PM to remove you.
* How can I deactivate Users?
*: Users can't be deactivated. They can only be removed from allocations or projects.
* My project or my allocation ist expired. How can I prolong it?
*: Please [mailto:sds-hd-support@urz.uni-heidelberg.de submit a Ticket]
* How can the PI of a project be changed?
*: This is currently not possible but will be possible in the future.

= Within the SV =
While bwServices-Management manages the access rights and storage amount on a top level outside of the storage space itself, there are multiple options to organize data and adjust access rights when being connected to the SV.

== Default Access Rights ==

* In the main space of an SV everyone has read and write access.
* The person who creates a file or folder is the owner. The owner can adjust the rights via [[Workspace#Regular_Unix_Permissions | chmod].
* The SV responsible has full control over the access rights by being able to take [[#Taking_Ownership_of_Files | ownership of files]].
* The SDS@hd support can help with defining access rules for your specific use case. Optimally, this is discussed before the SV gets filled with data. Examples:
** A group folder with read only access in contrast to the default read and write access. The SV members can see everything from each other but not change files that aren't their own.
** A users folder where every user can create their own folder with content that is inaccessible to others.

As an example, let's assume that the SV is structured as shown in the graphic below.
[[File:SDS_AccessRules_Overview.jpg|600px|center|Default access rules in a SV]]

In the SV scheme on the left you can see that the SV has two guests, two full members and the PI. The full members have read and write access to all normal folders in the SV root and the `guest` folder.

Additionally, there can be a `users` folder with a special setup. In the `users` folder everyone has read access but only the owners of the folders/files have write access.
<ins>Hint:</ins> If needed, the SDS@hd support can create custom access rules for specific top level folders.

The SV scheme on the right visualizes how the orange full member experiences the access when working in the SV. The full member has free read and write access to everything except the personal folder of the blue user.

== Guest Folder ==
Using the option to add some SV members with the role "Guest" allows better access management for separate user groups that shall not get access to the whole SV.

'''Use Cases:'''
* Data exchange with core facilities.
* Projects together with external collaborators.
* Up- and Download area

'''Setup:'''
* PMs can enable the guest folder by adding guests to the SV see [[#Guests | Guests]]. The folder gets created as soon as the first Guest user is defined. The creation might need up to 20 minutes.

'''Access Rights'''
* Full SV member:
** Read and write access to all content of the SV including the guest folder.
** No access to content that was deliberately made unaccessible by other SV members via <code>chmod</code>.
** No access to 'user folders'.
* Guest:
** Folder above guest folder:
*** Can see all file and folder names. Can't see content in subfolders.
*** Can't access or create content.
** Guest folder:
*** Can only create folders, no files, in the root of the guest folder.
*** Read content. Read content in subfolders.
**** Alternative setup: Read content. Can't read content in subfolders / folders from other guests. Please contact the sds-support to get this setup.
*** Has only write access to content that they created themselves.

The following graphic shows the current default access rights in the guest folder on the left in contrast to the alternative setup on the right.
[[File:SDS_AccessRules_Guest.jpg|600px|center|Guest folder default vs. custom]]

== Storage Space Management ==

At the moment you can't see how much storage space is left. We are working on including this information.

To keep the data hot, you should regularly check the following points:
* Make sure that temporary data gets removed.
* Make sure that no data from finished projects is stored. You might want to archive those with one of the archiving services listed on the [https://wiki.bwhpc.de/e/Main_Page Main Page].
* If someone has left the storage project, make sure that the PI has access to the data and the data is still hot.

SDS@hd/Registration

2026-09-02T10:39:04Z

H Schumacher: fixed links, added address / billing info

== Prerequisites ==

* Only scientific data is stored. Don't place administrative data on SDS@hd.
* Only hot data is stored which means only data that is actively used is allowed. We expect you to remove data from finished projects. It is advised to use an archiving service for this. Temporary data should also be removed on a regular basis.
* The storage of personal data or data capable of being linked to an individual is not permitted.
* SDS@hd mustn't be used as an archive or a backup. To find solutions in this regard, please have a look at the [https://wiki.bwhpc.de/e/Main_Page Main Page] and your institutional services.
* The use of SDS@hd must be adequately documented:
** All publications that were created with the involvement of the storage service SDS@hd should acknowledge its use. For an example text see [[SDS@hd/Acknowledgement | here]].
** These publications, scientific posters, theses and third-party funded projects for which the service was used need to be reported to us via [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] or via [mailto:sds-hd-support@urz.uni-heidelberg.de email].

== Registration Steps ==

The registration consists of two steps:

* '''Step A:''' Membership in Storage Project (Speichervorhaben/'''SV''', same as SDS@hd allocation). There are two options:
*: '''a)''' →[[#a)_Get_invited_to_existing_SV | Get invited to existing SV]]
*: '''b)''' →[[#b)_Apply_for_new_SV | Apply for new SV]]
* '''Step B:''' →[[#Step_B:_Registration_for_SDS@hd_Service |Registration for SDS@hd Service]]

After finishing the registration, the '''next steps''' are:
* Check which management options there are for your respective role in the SV → [[SDS@hd/SV_Management |SV Management]]
* See how to access and use your storage space → [[SDS@hd/Access | Access]]

[[File:Sds-hd-registration_new.png|center|x500px|upright|SDS@hd Registration Process]]

=== Step A: SV Membership ===
[[File:BwServices_Management_project_sv_structure.png|600px|thumb|right|A Project consists of multiple elements like a description, users and allocations/SVs. Every user in the project can be added to one or multiple of the available SVs.]]
You can either join an existing SV or start a new one. SVs are always part of a Project at the [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] page. In the image on the right you can see how the general setup of a Project looks like. The person who created the Project is called PI (Principal Investigator). The PI and the PMs (Project Managers), defined by the PI, can invite new users to the project and add them to SVs.

==== a) Get invited to existing SV ====

To join an existing SV as a coworker, ask the PI or a PM to add you to the project and afterwards to the SV.
* They can add you directly if you've already logged in to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] once.
* They can invite you by sending an email invite. You have to accept the invite to be a project member first, before you can be added to the SV.
Whenever a manager adds you to a project or SV, you get an information via email.

→ As soon as you're a member of an SV, you can [[#Step_B:_Registration_for_SDS@hd_Service | go on with Step B]] if you are using SDS@hd for the first time.

==== b) Apply for new SV ====

This is typically done only by the leader of a scientific work group or the senior scientist of a research group/collaboration.
You can add any amount of co-workers to your SV.

# '''Check prerequisites:''' Your institution must be a [https://www.bwidm.de/hochschulen.php bwIDM member] and you have to be an employee of your institution.
# '''Create a Project''' at [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] (alternatively use an already existing project):
## Go to the bwServices-Management Tool. At the start page click the title "Projects" and then "Add a project".
## Fill in and submit the form:
##* Keep in mind that this bwServices-Project can stand for a scientific research project but alternatively, it can for example represent all acitivities of a research group as well. The title and description can be changed later.
##* The information to fill in is as follows: <syntaxhighlight>
Please provide a detailed description of your work and select your field of science. In case that this Project corresponds to one dedicated scientific project, please describe your scientific project and indicate the expected project duration. Please mention additional infrastructure that is used by the project (cluster, cloud, storage, archiving services). Thank you!
</syntaxhighlight>
##* The provided address should be the one where a potential bill can be sent to. In cases where different SVs in the project would have different billing addresses, the address field in the SV application can be filled in additionally.
#: → Your project is directly active.
# '''Apply for SV'''. This is the same as adding allocations to a project:
## In the Allocations section of the Project's page is a button "Request Resource Allocation". Currently, the only allocation that can be chosen is an allocation for the resource SDS@hd. An SDS@hd allocation is called SV (Speichervorhaben / storage project).
## Fill in the form:
##* Justification questions (this field is the only one that can't be changed later on): <syntaxhighlight>
Q1: If not already provided in the bwServices-Management-Project description: Describe the intended usage of the storage space and indicate the expected duration that the allocation is needed.
Q2: How are data generated and processed? Please distinguish primary vs. secondary data and mention relevant equipment or software.
Q3: What is the expected data growth? Which file formats are used? Please describe any notable characteristics of your data, such as exceptionally large file size or very large number of small files.
Q4: Describe at which timepoints data can be archived or deleted. Describe the data and the data volume that can be archived at the end of the project.
Q5: How are your project data structured, and how is access managed? Are there special needs for the access rights within the SV (e.g. users or guest folder)?
</syntaxhighlight>
##* Title of the SV: The title is for you to recognize the SV in the allocations table if you have multiple SVs. It can be short.
## After filling in the form and agreeing to the terms, the allocation is shown on the project page with the status 'New'. A 'New' allocation is not approved, yet, and therefore not usable.
## The allocation application gets reviewed and approved. In the approval state no new users can be added. As soon as the storage space was set up, an email informs about the activation of the allocation.

→ [[#Step_B:_Registration_for_SDS@hd_Service | Go on with Step B]] if you are using SDS@hd for the first time.

If you're a PI in a project with SDS@hd allocations, you will be:
* ...held accountable for the co-workers in the SV
* ...responsible for all data stored in this SV, as well as for any costs incurred as a result.
* ...asked to provide information for the two reports required by the DFG for their funding of SDS@hd
* ...likely asked for a contribution to a future DFG grant proposal for an extension of the storage system in your area of research ("wissenschaftliches Beiblatt")

=== Step B: Registration for SDS@hd Service ===
[[File:bwServices_chooseHomeOrganisation.png|right|300px|thumb|Select your home organization]]

After step A you have to register your personal account on the storage system and set a service password.

* Visit the registration website '''bwServices''': [https://bwservices.uni-heidelberg.de/ https://bwservices.uni-heidelberg.de]
*# Select your home organization from the list and click ''Proceed''
*#: You will be directed to the ''Identity Provider'' of your home organisation
*# Enter your home-organisational user ID / username and your home-organisational password and click ''Login'' button
*#: You will be redirected back to bwServices
* '''Register''' for the SDS@hd service:
*# Under ''The following services are available'' select the service ''SDS@hd - Scientific Data Storage''.
*# Click ''Register''
* Set your personal '''service password''' for SDS@hd. This is used for accessing SVs.
* The registration is finished.
*: → [[#sds_registration_nextSteps|next steps]]

=== Change Service Password at bwServices ===

→ '''[[Registration/bwForCluster/Helix#Setting_a_New_Service_Password | Setting a New Service Password]]'''

== FAQ ==
* Is there a Test-SV to do technical tests?
*: Yes, there is a Test-SV for this purpose. You can write to the support to get invited. Please keep in mind that data gets regularly deleted and users may be removed after some time.
* I can login with different institutions/IDs (edugain, ORCID, university ID, university clinics ID,...). Which one shall I choose?
*: If you want to be able to start your own SVs, use the ID were you can get the entitlement most conveniently. Otherwise it doesn't matter. But ORCID is the most unspecific option and therefore should only be chosen if the other options don't work.
* I own a SV with my ID at institution A. Can I connect my account from institution B to it?
*: You can't connect different accounts. But you can join the SV with both accounts if necessary.
* What is my userID when I register via ORCID?
*: You'll be informed about the ID along the process. It'll start with "orcid_".
* How do I set up ORCID?
*: Login with your ORCID account (instead of an institutional account) and set it up so that your email address is shared with us.
* How do I register with foreign institutions (outside of [https://www.belwue.de/ueberuns/teilnehmer.html BelWü])
*: We might need to enable the IP range first.
*: After we've added the ip addresses, you might need to wait for an hour before it works.

Main Page

2026-09-01T15:18:44Z

H Schumacher: updated conditions for opening an SV on SDS@hd

'''Welcome to the bwHPC Wiki.'''

bwHPC represents services and resources in the State of '''B'''aden-'''W'''ürttemberg, Germany, for High Performance Computing ('''HPC'''), Data Intensive Computing ('''DIC''') and Large Scale Scientific Data Management ('''LS2DM''').

The main bwHPC web page is at '''[https://www.bwhpc.de/ https://www.bwhpc.de/]''' including the latest [https://www.bwhpc.de/news.html News] and [https://www.bwhpc.de/projects.php Success Stories].

Many topics depend on the cluster system you use.
First choose the cluster you use, then select the correct topic.








{| style=" background:#eeeefe; width:100%;"
| style="padding:8px; background:#dedefe; font-size:120%; font-weight:bold; text-align:left" | Courses / eLearning
|-
|
* [https://training.bwhpc.de/ eLearning and Online Courses]
* [https://hpc-wiki.info/hpc/Introduction_to_Linux_in_HPC Introduction to Linux in HPC (external resource)]
|}

{| style=" background:#deffee; width:100%;"
| style="padding:8px; background:#cef2e0; font-size:120%; font-weight:bold; text-align:left" | [[File:Clusternodes.svg|x20px]] Need Access to a Cluster?
|-
|
* [[When to use an HPC Cluster]]
* [[Running Calculations]]
* [[Registration]]
|}

{| style=" background:#deffee; width:100%;"
| style="padding:8px; background:#cef2e0; font-size:120%; font-weight:bold; text-align:left" | [[File:Clusternodes.svg|x20px]] HPC System Specific Documentation
|-
|
bwHPC encompasses several HPC compute clusters at different universities in Baden-Württemberg. Each cluster is dedicated to [https://www.bwhpc.de/bwhpc-domains.php specific research domains].

Documentation differs between compute clusters, please see cluster specific overview pages:
{|
|-
| style="padding:5px; width:30%" | [[BwUniCluster3.0|bwUniCluster 3.0]]
| style="padding-left:20px;" | General Purpose, Teaching
|-
| style="padding:5px; width:30%" | [[BinAC2|bwForCluster BinAC 2]]
| style="padding-left:20px;" | Bioinformatics, Astrophysics, Geosciences, Pharmacy, and Medical Informatics
|-
| style="padding:5px; width:30%" | [[Helix|bwForCluster Helix]]
| style="padding-left:20px;" | Structural and Systems Biology, Medical Science, Soft Matter, Computational Humanities, and Mathematics and Computer Science
|-
| style="padding:5px; width:30%" | [[:JUSTUS2| bwForCluster JUSTUS 2]]
| style="padding-left:20px;" | Theoretical Chemistry, Condensed Matter Physics, and Quantum Sciences
|-
| style="padding:5px; width:30%" | [[NEMO2|bwForCluster NEMO 2]]
| style="padding-left:20px;" | Neurosciences, Particle Physics, Materials Science, and Microsystems Engineering
|}
|-
|bwHPC Clusters: [https://www.bwhpc.de/cluster.php operational status]
Further Compute Clusters in Baden-Württemberg (separate access policies):
* [[DACHS | Datenanalyse Cluster der Hochschulen (DACHS)]]
* bwHPC tier 1: [https://kb.hlrs.de/platforms/index.php/Hunter_(HPE) Hunter] ([https://www.hlrs.de/apply-for-computing-time getting access])
* bwHPC tier 2: [https://www.nhr.kit.edu/userdocs/horeka HoreKa] ([https://www.nhr.kit.edu/userdocs/horeka/projects/ getting access])
|}

{| style="background:#deffee; width:100%;"
| style="padding:8px; background:#cef2e0; font-size:120%; font-weight:bold; text-align:left" | [[File:Clusternodes.svg|x20px]] Documentation valid for all Clusters
|-
|
* [[Environment Modules| Software Environment Modules]]: usage of preexisting software
* [https://www.bwhpc.de/software.html List of Software] on all clusters
* [[Data Transfer|Data Transfer]]: working with files [[File:Notebook.svg|x15px]] <---> [[File:Clusternodes.svg|x15px]]
* [[Development| Development]]: software, programming languages, parallel pogramming
* [[Energy Efficient Cluster Usage | (Energy) Efficient Cluster Usage]] for low waiting times and fast running jobs
* [[HPC Glossary]]

{| style="height:100%; background:#ffeaef; width:100%"
| style="padding:8px; background:#f5dfdf; font-size:120%; font-weight:bold; text-align:left" | [[File:Storage_small.svg|x15px]] Scientific Data Storage
|-
|
User guides of available scientific data storage services:
* [[SDS@hd]]: Everyone can join existing storage projects. New projects and SVs can only be requested by bwIDM members with employee status. Costs can arise, see [https://sds-hd.urz.uni-heidelberg.de/management/shib/sds_costs.php cost sheet].
* [https://uni-tuebingen.de/einrichtungen/zentrum-fuer-datenverarbeitung/projekte/laufende-projekte/bwsfs bwSFS]
Associated, but local scientific storage services are:
* [https://wiki.scc.kit.edu/lsdf/index.php/Category:LSDF_Online_Storage LSDF Online Storage] (only for KIT and KIT partners)
|}

{| style="height:100%; background:#ffeaef; width:100%"
| style="padding:8px; background:#f5dfdf; font-size:120%; font-weight:bold; text-align:left" | [[File:Storage_small.svg|x15px]] Scientific Data Archiving
|-
|
For user guides of the scientific data archiving services:
* [https://www.rda.kit.edu/english bwDataArchive]: Open to all scientists at KIT and institutions in Baden-Württemberg that have signed a service agreement with KIT
Associated, but local archiving services for scientific data are:
* [https://www.urz.uni-heidelberg.de/de/service-katalog/speicher/heiarchive heiARCHIVE] (only for members of Heidelberg University)
For instructions on how to move the data, depending on the offered connections, can be found under [[Data Transfer | Data Transfer]].
|}

{| style="height:100%; background:#ffeaef; width:100%"
| style="padding:8px; background:#f5dfdf; font-size:120%; font-weight:bold; text-align:left" | Research Data Management
|-
|
* [https://www.forschungsdaten.org/index.php/FDM-Kontakte#Deutschland Research Data Management (RDM)] contact persons
* [https://www.forschungsdaten.info Portal for Research Data Management] (Forschungsdaten.info)
|}
{| style=" background:#eeeefe; width:100%;"
| style="padding:8px; background:#dedefe; font-size:120%; font-weight:bold; text-align:left" | Support
|-
|
* [[BwSupportPortal|Submit a Ticket in our Support Portal]]
Support is provided by the [https://www.bwhpc.de/teams.php bwHPC Competence Centers]:
|}
{| style=" background:#e6e9eb; width:100%;"
| style="padding:8px; background:#d1dadf; font-size:120%; font-weight:bold; text-align:left" | Acknowledgement
|-
|
* Please [[Acknowledgement|acknowledge]] our resources in your publications.
|}

SDS@hd

2026-09-01T15:14:23Z

H Schumacher: added news: bwServices-Management

[[File:sds-hd-logo.png|350px]]

SDS@hd is a central service for securely storing scientific data (Scientific Data Storage). The service is provided as a state service to researchers of higher education institutions of Baden-Württemberg. It is intended to be used for data that is frequently accessed ('hot data').

{| style=" background:#FEF4AB; width:100%;"
| style="padding:8px; background:#FFE856; font-size:120%; font-weight:bold; text-align:left" | News
|-
|
* November 2022: No special entitlement is needed anymore to participate in an existing storage project.
* September 2023: Service has now been opened for DFN AAI & eduGAIN federation members to participate on existing storage projects.
* September 2026: Migration from 'SDS@hd Management' to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] for improved registration and SV management processes.
|}

{| style=" background:#eeeefe; width:100%;"
| style="padding:8px; background:#dedefe; font-size:120%; font-weight:bold; text-align:left" | Training & Support
|-
|
* [https://www.urz.uni-heidelberg.de/en/service-catalogue/storage/sdshd-scientific-data-storage Service description & FAQ]
* [mailto:sds-hd-support@urz.uni-heidelberg.de Submit a Ticket]
|}

{| style=" background:#ffeaef; width:100%;"
| style="padding:8px; background:#f5dfdf; font-size:120%; font-weight:bold; text-align:left" | User Documentation
|-
|
* [[SDS@hd/Registration|Registration]] ([[SDS@hd/SV_Management|SV Management]], [[Registration/bwForCluster/Helix#Registration_Website_bwServices|Account Management]])
* [[SDS@hd/Access|Access]]
* [[SDS@hd/Security|Data Security]]
* [[Helix/bwVisu|bwVisu]] for visualization and interactive data analysis
|}

{| style=" background:#e6e9eb; width:100%;"
| style="padding:8px; background:#d1dadf; font-size:120%; font-weight:bold; text-align:left" | Storage Funding
|-
|
* Please [[SDS@hd/Acknowledgement|acknowledge]] SDS@hd in your publications.

SDS@hd/Registration

2026-09-01T15:14:21Z

H Schumacher: update to bwServices-Management

== Prerequisites ==

* Only scientific data is stored. Don't place administrative data on SDS@hd.
* Only hot data is stored which means only data that is actively used is allowed. We expect you to remove data from finished projects. It is advised to use an archiving service for this. Temporary data should also be removed on a regular basis.
* The storage of personal data or data capable of being linked to an individual is not permitted.
* SDS@hd mustn't be used as an archive or a backup. To find solutions in this regard, please have a look at the [https://wiki.bwhpc.de/e/Main_Page Main Page] and your institutional services.
* The use of SDS@hd must be adequately documented:
** All publications that were created with the involvement of the storage service SDS@hd should acknowledge its use. For an example text see [[SDS@hd/Acknowledgement | here]].
** These publications, scientific posters, theses and third-party funded projects for which the service was used need to be reported to us via [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] or via [mailto:sds-hd-support@urz.uni-heidelberg.de email].

== Registration Steps ==

The registration consists of two steps:

* '''Step A:''' Membership in Storage Project (Speichervorhaben/'''SV''', same as SDS@hd allocation). There are two options:
*: '''a)''' →[[SDS@hd/Registration_New#a)_Get_invited_to_existing_SV | Get invited to existing SV]]
*: '''b)''' →[[SDS@hd/Registration_New#b)_Apply_for_new_SV | Apply for new SV]]
* '''Step B:''' →[[SDS@hd/Registration_New#Step_B:_Registration_for_SDS@hd_Service |Registration for SDS@hd Service]]

After finishing the registration, the '''next steps''' are:
* Check which management options there are for your respective role in the SV → [[SDS@hd/SV_Management_New |SV Management]]
* See how to access and use your storage space → [[SDS@hd/Access | Access]]

[[File:Sds-hd-registration_new.png|center|x500px|upright|SDS@hd Registration Process]]

=== Step A: SV Membership ===
[[File:BwServices_Management_project_sv_structure.png|600px|thumb|right|A Project consists of multiple elements like a description, users and allocations/SVs. Every user in the project can be added to one or multiple of the available SVs.]]
You can either join an existing SV or start a new one. SVs are always part of a Project at the [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] page. In the image on the right you can see how the general setup of a Project looks like. The person who created the Project is called PI (Principal Investigator). The PI and the PMs (Project Managers), defined by the PI, can invite new users to the project and add them to SVs.

==== a) Get invited to existing SV ====

To join an existing SV as a coworker, ask the PI or a PM to add you to the project and afterwards to the SV.
* They can add you directly if you've already logged in to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] once.
* They can invite you by sending an email invite. You have to accept the invite to be a project member first, before you can be added to the SV.
Whenever a manager adds you to a project or SV, you get an information via email.

→ As soon as you're a member of an SV, you can [[SDS@hd/Registration_New#Step_B:_Registration_for_SDS@hd_Service | go on with Step B]] if you are using SDS@hd for the first time.

==== b) Apply for new SV ====

This is typically done only by the leader of a scientific work group or the senior scientist of a research group/collaboration.
You can add any amount of co-workers to your SV.

# '''Check prerequisites:''' Your institution must be a [https://www.bwidm.de/hochschulen.php bwIDM member] and you have to be an employee of your institution.
# '''Create a Project''' at [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] (alternatively use an already existing project):
## Go to the bwServices-Management Tool. At the start page click the title "Projects" and then "Add a project".
## Fill in and submit the form.
##* Keep in mind that this Project can stand for a scientific research project but alternatively, it can for example represent all acitivities of a research group as well. The title and description can be changed later.
##* The information to fill in is as follows: <syntaxhighlight>
Please provide a detailed description of your work and select your field of science. In case that this Project corresponds to one dedicated scientific project, please describe your scientific project and indicate the expected project duration. Please mention additional infrastructure that is used by the project (cluster, cloud, storage, archiving services). Thank you!
</syntaxhighlight>
#: → Your project is directly active.
# '''Apply for SV'''. This is the same as adding allocations to a project:
## In the Allocations section of the Project's page is a button "Request Resource Allocation". Currently, the only allocation that can be chosen is an allocation for the resource SDS@hd. An SDS@hd allocation is called SV (Speichervorhaben / storage project).
## Fill in the form:
##* Justification questions (this field is the only one that can't be changed later on): <syntaxhighlight>
Q1: If not already provided in the bwServices-Management-Project description: Describe the intended usage of the storage space and indicate the expected duration that the allocation is needed.
Q2: How are data generated and processed? Please distinguish primary vs. secondary data and mention relevant equipment or software.
Q3: What is the expected data growth? Which file formats are used? Please describe any notable characteristics of your data, such as exceptionally large file size or very large number of small files.
Q4: Describe at which timepoints data can be archived or deleted. Describe the data and the data volume that can be archived at the end of the project.
Q5: How are your project data structured, and how is access managed? Are there special needs for the access rights within the SV (e.g. users or guest folder)?
</syntaxhighlight>
##* Title of the SV: The title is for you to recognize the SV in the allocations table if you have multiple SVs. It can be short.
## After filling in the form and agreeing to the terms, the allocation is shown on the project page with the status 'New'. A 'New' allocation is not approved, yet, and therefore not usable.
## The allocation application gets reviewed and approved. In the approval state no new users can be added. As soon as the storage space was set up, an email informs about the activation of the allocation.

→ [[SDS@hd/Registration_New#Step_B:_Registration_for_SDS@hd_Service | Go on with Step B]] if you are using SDS@hd for the first time.

If you're a PI in a project with SDS@hd allocations, you will be:
* ...held accountable for the co-workers in the SV
* ...responsible for all data stored in this SV, as well as for any costs incurred as a result.
* ...asked to provide information for the two reports required by the DFG for their funding of SDS@hd
* ...likely asked for a contribution to a future DFG grant proposal for an extension of the storage system in your area of research ("wissenschaftliches Beiblatt")

=== Step B: Registration for SDS@hd Service ===
[[File:bwServices_chooseHomeOrganisation.png|right|300px|thumb|Select your home organization]]

After step A you have to register your personal account on the storage system and set a service password.

* Visit the registration website '''bwServices''': [https://bwservices.uni-heidelberg.de/ https://bwservices.uni-heidelberg.de]
*# Select your home organization from the list and click ''Proceed''
*#: You will be directed to the ''Identity Provider'' of your home organisation
*# Enter your home-organisational user ID / username and your home-organisational password and click ''Login'' button
*#: You will be redirected back to bwServices
* '''Register''' for the SDS@hd service:
*# Under ''The following services are available'' select the service ''SDS@hd - Scientific Data Storage''.
*# Click ''Register''
* Set your personal '''service password''' for SDS@hd. This is used for accessing SVs.
* The registration is finished.
*: → [[#sds_registration_nextSteps|next steps]]

=== Change Service Password at bwServices ===

→ '''[[Registration/bwForCluster/Helix#Setting_a_New_Service_Password | Setting a New Service Password]]'''

== FAQ ==
* Is there a Test-SV to do technical tests?
*: Yes, there is a Test-SV for this purpose. You can write to the support to get invited. Please keep in mind that data gets regularly deleted and users may be removed after some time.
* I can login with different institutions/IDs (edugain, ORCID, university ID, university clinics ID,...). Which one shall I choose?
*: If you want to be able to start your own SVs, use the ID were you can get the entitlement most conveniently. Otherwise it doesn't matter. But ORCID is the most unspecific option and therefore should only be chosen if the other options don't work.
* I own a SV with my ID at institution A. Can I connect my account from institution B to it?
*: You can't connect different accounts. But you can join the SV with both accounts if necessary.
* What is my userID when I register via ORCID?
*: You'll be informed about the ID along the process. It'll start with "orcid_".
* How do I set up ORCID?
*: Login with your ORCID account (instead of an institutional account) and set it up so that your email address is shared with us.
* How do I register with foreign institutions (outside of [https://www.belwue.de/ueberuns/teilnehmer.html BelWü])
*: We might need to enable the IP range first.
*: After we've added the ip addresses, you might need to wait for an hour before it works.

SDS@hd/SV Management

2026-09-01T15:14:18Z

H Schumacher: update to bwServices-Management

{| style=" background:#FEF4AB; width:100%;"
| style="padding:8px; background:#FFE856; font-size:120%; text-align:left" | For (previous) users of the SDS@hd Management Tool:
|-
|
On the 1st of September we're going to replace the SDS@hd Management website with the [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] website. 
The '''main changes in the registration and management process''' are as follows:
* An SV must be a part of a bwServices-Project.
* An SV is an ''Allocation'' requested from the ''Resource'' SDS@hd.
* The SVV role was removed. SVVs are now the PI (Principal Investigator) of a bwServices-Project that contains the SV.
* The SV-Manager role was removed. Managers (PMs) can be assigned within the project and have mostly the same rights as the PI.
* Publications can be added to the project at any time.
* New SV members can directly be added by one of the PMs. There is no need for sharing an SV-password and filling in a collaboration form anymore.
* New SV members are always added as 'Users' (= full members). If you want to add guests, please directly change their role to 'Guest' after adding them!

'''Important points for the migration:'''
The data of current SVs were migrated to the new page:
* The SVV is now the PI of a project and the project has an SDS@hd allocation.
* Inactive members of the SVs were removed.
* <ins>Initial tasks for the PIs</ins>:
*: Login to [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] and update the following points:
** In the project:
*** Check and update the data (project description and attributes).
*** Define managers. Read their role description first! Check their rights in the [[#Project_Manager_(PM) | context of the project]] and in [[#PMs | context of the allocation]] (SV).
** In the allocation/SV:
*** Check the members list and if applicable the guest roles.
*** Check the attributes of the allocation (access options, quota, ...). They can be changed via [[#Adjust_SV_Attributes | Change Request]].
*** Billing data must only be provided if the SV generates costs according to the [https://sds-hd.urz.uni-heidelberg.de/management/shib/en/sds_costs_download.php price sheet] and if additionally, the billing information is different from the information provided in the project.
* The SDS@hd Management page will stay in read-only mode for a short while before it gets deactivated.
* Test-SV: Members that weren't active in the last 30 days were removed. Access to the Test-SV is given on request to the SDS@hd Support.
* Inactive PI: We'll get in contact with projects that don't have an active PI.
|}
This page provides an overview of options to structure projects that use SDS@hd. The simple use case is:
* One research project has one bwServices-Management Project (in short bwServices-Project).
* One bwServices-Project has one SV.
* Both have the same group of members.
The graphic below visualizes one option for another setup. Depending on your preferences of who should have access to which data, you can structure things differently.
[[File:BwServices_Management_project_sv_structure.png|600px|thumb|right|A Project consists of multiple elements like a description, users and allocations/SVs. Every user in the project can be added to one or multiple of the available SVs.]]

= bwServices-Management =

In bwServices-Management the top level entity is a project. On this page ''project'' always refers to this ''bwServices-Project''. Every bwIDM member with employee status can create a project by going to the ''Projects'' page (Home -> Projects) and then clicking on the "Add a project" button. A project can have multiple users. For specifics regarding the different roles in a project see the [[#Roles_in_a_Project| Roles]] section.
[[File:bwServices_Management_home.png|300px|thumb|right|The start page lists recent projects and allocations. The titles link to the complete lists.]]

'''Adding users to a project:'''
# On the project's page either the button on the top or next to the users list can be used for adding users.
# There are two ways to add new project members:
#:* Option 1: To invite someone that already logged in to the bwServices-Management page once, they can be found...
#:** ... via their user id. A list of ids can be provided in the search field.
#:** ... by providing (a part of) the name and searching "All Fields".
#:* Option 2: Alternatively, an invite link can be sent by providing the email address. The email address for sending the invite doesn't need to be the address that is later on used for their account on bwServices-Management. When a user clicks on the invite link they have to login to bwServices-Management. Afterwards, they are a part of the project with the account that they logged in with.
# Before adding the person or submitting the invite, the role of the new project member should be adjusted if needed. Available roles are "[[#User |User]]" or "[[#Project_Manager_(PM) |Manager]]".
# The added person gets an email notification regarding their new project membership.

A project can have multiple allocations. Having an allocation means that you can allocate a resource like SDS@hd for your project. An allocation comes with its own form to fill in and its own terms to agree to. You can find detailed instructions under [[SDS@hd/Registration_New#b)_Apply_for_new_SV | Registration/Apply for new SV]].

'''Adding users to an allocation:'''

An allocation can have multiple users. The users can be chosen from the list of project members.

Here is an overview of the most important pages and what you can find there:
[[File:bwServices_Management_pagetree.png|600px|center|bwServices-Management Page Overview]]

== Roles in a Project ==
There are three different roles in a project. Depending on the role, different management activites can be executed at [https://management.bwservices.uni-heidelberg.de/ bwServices-Management]. These are described below. They include top level management of user access.

=== User ===
Users can...
* ...view project information.
* ...view allocations that they are a member of.

* ...enable/disable notifications. Notifications regarding the Projects that they are (or will be) a member of can be:
** information that they were added to the project
** emails from one of the project's PMs
** information about a project being archived

=== Project Manager (PM) ===
Additionally to the abilities of a User, a PM can...
* ...add and remove Users from the project.
* ...add and remove Users from allocations.
* ...change the roles of Users on project and allocation level (Project: User <-> PM; SV: None <-> Guest).

* ...enable/disable notifications for other Users.
* ...always receive email notifications.
* ...write an email to all Project Users.

* ...request new allocations.
* ...change project attributes.
* ...add publications.
* ...extend project lifetime.
* ...extend allocation lifetime.
* ...can archive (~ delete) the project.

==== Yearly Project Extension ====

Once a year a PM in the project needs to apply for a lifetime extension. This is indicated by the label "Project Review". Please update your project information first before applying for the extension. Especially, update the list of publications for which the allocated resources (e.g. SDS@hd) were used.

==== Archiving a Project ====
Caution: Same as '''deleting''' all allocations!

For PMs there is a button for archiving the project in the upper right part of the project page. Archiving means that the status of the allocations changes to "expired" and the [[#SV_Deprovisioning | deprovisioning]] process starts. The project with its description will be deprovisioned some time after all allocations are removed.

=== Principal Investigator (PI) ===
A PI is automatically a PM as well. -> Mentions of 'PM' always include the PI.
Additionally to the abilities of a PM, the PI...
* ...is responsible for the whole project. This includes all allocations, independently of who requested them.
* ...is automatically a member of every allocation.

== Roles in a SV ==

The SV users keep their project roles (PI/PM/User). Additionally, they can hold the Guest role in an SV. The roles within the context of an SV are described below.

=== All SV Users ===
At [https://management.bwservices.uni-heidelberg.de/ bwServices-Management] they can...
* ...view their role in the SV. If no Role column is shown, there are no users with a Guest role in the SV.
* ...view the status of the SV (active/expired, end date)
Notifications regarding the SVs that they are a member of can be:
** information about having been added to an allocation
** information about accepted or denied change requests (without the concrete content of the request)
** information about an allocation being activated or denied

=== Guests ===
PMs can add the "Guest" role to a user in the SV by going to the users list of the allocation and clicking the pencil button in the last column of the user's row. This leads to the details of the user where the role can be changed.
As soon as the first user is defined as Guest via bwServices-Management, a "guest" folder is created in the SV. A Guest can see but not access content in the SV root. They can only work within the "guest" folder. For more information see [[#Guest_Folder | Guest Folder]].
'''Caution:''' When adding new guests to the Project and afterwards to the SV, make sure to directly change their role in the SV from the default to "Guest". Otherwise they have full access first.

=== PMs ===
PMs in the project can...
* ...manage the SV as mentioned in their role description.
* ...adjust SV attributes via change request. Previous change requests can be viewed.
* ...extend the SV lifetime.
* ...request the deletion of the SV.

==== Adjust SV Attributes ====

To change the properties of your SV, you can hand in a Change Request.
When you request a quota expansion for your SV at the SDS@hd service in order to store hot scientific data, we kindly ask you to answer the questions in the justification textfield for reasoning the extension. The questions are:
<syntaxhighlight>Q1: Please explain why additional storage is needed.
Q2: What is the potential for deleting or archiving data?</syntaxhighlight>

Please also consider the following remarks:
Upon increasing storage capacities, there will also be a need for further activities in the reporting from your side, e.g. by pointing out the usage of SDS@hd in scientific publications to us. More information about this can be found here: https://wiki.bwhpc.de/e/SDS@hd/Acknowledgement
Please consider raising funds for the usage and further operation of the SDS@hd service. We will gladly support you with this.

==== Yearly SV Extension ====

Once a year a PM needs to apply for a lifetime extension of the storage project. This is indicated by the label "Allocation Renewal". As long as SDS@hd is used appropriately, an extension will be granted.
Warning: You have to apply for the extension in the given timeframe. When you're too late, you have to contact the sds-support to get the extension. SVs that don't get extended, are going to be deleted by us, see [[#SV_Deprovisioning | SV Deprovisioning]]. Before the deletion, the PI is informed via email.

==== SV Deprovisioning ====
The deprovisioning is initiated in the following cases:
* The SV is not extended.
* The Project was archived.
* A PM that is a member of the SV asks via Change Request for a deletion of the SV by writing this request in the "Justification for Changes" field.

The deprovisioning basically contains the following steps:
* The access of the SV-group to the SV is blocked. The access to any data that is saved within the SV is not possible anymore.
* After a specific period of time all data in the SV gets permanently deleted.

== FAQ ==

* I changed a user's role in bwServices-Management but the user still has the previous access rights.
*: After 30 minutes the adjustment should be synced to all systems. If not, it helps when the member with the role change logs in to bwServices once to trigger an automatic update and then waits for 30 minutes.
* How can I remove myself from an allocation or project?
*: This is not possible. You must ask a PM to remove you.
* How can I deactivate Users?
*: Users can't be deactivated. They can only be removed from allocations or projects.
* My project or my allocation ist expired. How can I prolong it?
*: Please [mailto:sds-hd-support@urz.uni-heidelberg.de submit a Ticket]
* How can the PI of a project be changed?
*: This is currently not possible but will be possible in the future.

= Within the SV =
While bwServices-Management manages the access rights and storage amount on a top level outside of the storage space itself, there are multiple options to organize data and adjust access rights when being connected to the SV.

== Default Access Rights ==

* In the main space of an SV everyone has read and write access.
* The person who creates a file or folder is the owner. The owner can adjust the rights via [[Workspace#Regular_Unix_Permissions | chmod].
* The SV responsible has full control over the access rights by being able to take [[#Taking_Ownership_of_Files | ownership of files]].
* The SDS@hd support can help with defining access rules for your specific use case. Optimally, this is discussed before the SV gets filled with data. Examples:
** A group folder with read only access in contrast to the default read and write access. The SV members can see everything from each other but not change files that aren't their own.
** A users folder where every user can create their own folder with content that is inaccessible to others.

As an example, let's assume that the SV is structured as shown in the graphic below.
[[File:SDS_AccessRules_Overview.jpg|600px|center|Default access rules in a SV]]

In the SV scheme on the left you can see that the SV has two guests, two full members and the PI. The full members have read and write access to all normal folders in the SV root and the `guest` folder.

Additionally, there can be a `users` folder with a special setup. In the `users` folder everyone has read access but only the owners of the folders/files have write access.
<ins>Hint:</ins> If needed, the SDS@hd support can create custom access rules for specific top level folders.

The SV scheme on the right visualizes how the orange full member experiences the access when working in the SV. The full member has free read and write access to everything except the personal folder of the blue user.

== Guest Folder ==
Using the option to add some SV members with the role "Guest" allows better access management for separate user groups that shall not get access to the whole SV.

'''Use Cases:'''
* Data exchange with core facilities.
* Projects together with external collaborators.
* Up- and Download area

'''Setup:'''
* PMs can enable the guest folder by adding guests to the SV see [[#Guests | Guests]]. The folder gets created as soon as the first Guest user is defined. The creation might need up to 20 minutes.

'''Access Rights'''
* Full SV member:
** Read and write access to all content of the SV including the guest folder.
** No access to content that was deliberately made unaccessible by other SV members via <code>chmod</code>.
** No access to 'user folders'.
* Guest:
** Folder above guest folder:
*** Can see all file and folder names. Can't see content in subfolders.
*** Can't access or create content.
** Guest folder:
*** Can only create folders, no files, in the root of the guest folder.
*** Read content. Read content in subfolders.
**** Alternative setup: Read content. Can't read content in subfolders / folders from other guests. Please contact the sds-support to get this setup.
*** Has only write access to content that they created themselves.

The following graphic shows the current default access rights in the guest folder on the left in contrast to the alternative setup on the right.
[[File:SDS_AccessRules_Guest.jpg|600px|center|Guest folder default vs. custom]]

== Storage Space Management ==

At the moment you can't see how much storage space is left. We are working on including this information.

To keep the data hot, you should regularly check the following points:
* Make sure that temporary data gets removed.
* Make sure that no data from finished projects is stored. You might want to archive those with one of the archiving services listed on the [https://wiki.bwhpc.de/e/Main_Page Main Page].
* If someone has left the storage project, make sure that the PI has access to the data and the data is still hot.

File:SDS AccessRules Overview.jpg

2026-09-01T14:53:29Z

H Schumacher: H Schumacher uploaded a new version of File:SDS AccessRules Overview.jpg

File:SDS AccessRules Guest.jpg

2026-09-01T14:47:01Z

H Schumacher: H Schumacher uploaded a new version of File:SDS AccessRules Guest.jpg

Python

2026-08-31T08:45:13Z

H Schumacher: added information

Overview of pages related to the programming language Python.
Keep in mind that software modules like python always have documentation directly on the cluster as well (module help).

→ '''[[Development/Python | Python Main Page]]'''

'''Additional cluster-specific pages:'''
* [[JUSTUS2/Software/Python | Python on JUSTUS 2]]

*''Instructions for JupyterLab on...''
** [[BwUniCluster3.0/Jupyter | bwUniCluster3.0 ]]
** [[Helix/bwVisu/JupyterLab | Helix]] (through [[Helix/bwVisu | bwVisu]])
** [[BinAC2/Software/Jupyterlab | BinAC2]] (as software module)
** [[Development/VS_Code#Connect_to_Remote_Jupyter_Kernel | Other]]

'''Related software packages'''
* [[BwUniCluster3.0/Software/Python Dask | Dask]] (on bwUniCluster3.0)
* [[Development/ollama | ollama ]]

'''Training'''

* [https://github.com/hpcraink/workshop-parallel-jupyter Python Workshop]
** Python Basics
** Using parallel code in jobs
** Machine Learning

File:Sds-hd-registration new.png

2026-08-17T11:51:21Z

H Schumacher: H Schumacher uploaded a new version of File:Sds-hd-registration new.png

File:SDS@hd Registration.png

2026-08-17T11:50:48Z

H Schumacher: H Schumacher reverted File:SDS@hd Registration.png to an old version

File:SDS@hd Registration.png

2026-08-17T11:49:14Z

H Schumacher: H Schumacher uploaded a new version of File:SDS@hd Registration.png

File:SDS AccessRules Overview.jpg

2026-08-17T07:13:44Z

H Schumacher:

File:SDS AccessRules Guest.jpg

2026-08-17T07:13:27Z

H Schumacher:

SDS@hd/Access

2026-08-14T09:19:58Z

H Schumacher: added smb capable institutions

This page provides an overview on how to access data served by SDS@hd. To get an introduction to data transfer in general, see [[Data_Transfer|data transfer]].

'''Notation:''' Please replace all placeholder contents within guillemets <code>< ></code> with the actual value. For example, replace ''<name_of_my_dog>'' with ''Fluffy'' .

== Prerequisites ==

* You need to be [[SDS@hd/Registration|registered]].
* You need to be in the belwue-Network. This means you have to use the VPN Service of your HomeOrganization, if you want to access SDS@hd from outside the bwHPC-Clusters (e.g. via eduroam or from your personal notebook).

== Needed Information, independent of the chosen tool ==

* '''Username:''' You can look it up under your user data at [https://bwservices.uni-heidelberg.de/user/index.xhtml bwServices]. The username is ''<localPrimaryGroup>_<eduPersonPrincipalName>'' . (It is the same as for the bwHPC Clusters. Example: ''hd_ab123''.)
* '''Password:''' The Service Password that you set at bwServices in the [[SDS@hd/Registration#Step_B:_Registration_for_SDS@hd_Service|registration step]].
* '''SV-Acronym:''' Use the lower case version of the acronym for all access options.
* '''Hostname:''' The hostname depends on the chosen network protocol:
** For [[Data_Transfer/SSHFS|SSHFS]] and [[Data_Transfer/SFTP|SFTP]]: ''lsdf02-sshfs.urz.uni-heidelberg.de''
** For [[SDS@hd/Access/SMB|SMB]] and [[SDS@hd/Access/NFS|NFS]]: ''lsdf02.urz.uni-heidelberg.de''
** For [[Data_Transfer/WebDAV|WebDAV]] the url is: ''https://lsdf02-webdav.urz.uni-heidelberg.de''
* '''Domain:''' The domain is ''BWSERVICESAD'' (needed for smb and nfs connections).

== Recommended Setup ==
The following graphic shows the recommended way for accessing SDS@hd via Windows/Mac/Linux. The table provides an overview of the most important access options and links to the related pages. 
If you have various use cases, it is recommended to use [[Data_Transfer/Rclone|Rclone]]. You can copy, sync and mount with it. Thanks to its multithreading and checksumming capability Rclone is a good fit for transferring big data. 
For an overview of all connection possibilities, please have a look at [[Data_Transfer/All_Data_Transfer_Routes|all data transfer routes]].

[[File:Data_transfer_diagram_simple.jpg|center|700px]]
Figure 1: SDS@hd main transfer routes
*Institutions that allow smb connections to SDS@hd are for example: Heidelberg University, Karlsruhe Institute of Technology (KIT), Hohenheim University, Freiburg University, Mannheim University, Hochschule Rottenburg, Hochschule Biberach.

{| class="wikitable"
|- style="font-weight:bold; text-align:center; vertical-align:middle;"
!
! Use Case
! Windows
! Mac
! Linux
! Possible Bandwith
! Firewall Ports
|-
| [[Data_Transfer/Rclone|Rclone]] + <protocol>
| copy, sync and mount, multithreading, checksumming for verifying success of up-/download
| ✓
| ✓
| ✓
| depends on used protocol
| depends on used protocol
|-
| [[SDS@hd/Access/SMB|SMB]]
| mount as network drive in file explorer or usage via Rclone
| [[SDS@hd/Access/SMB#Windows|✓]]
| [[SDS@hd/Access/SMB#Mac|✓]]
| [[SDS@hd/Access/SMB#Linux|✓]]
| up to 40 Gbit/sec
| 139 (netbios), 135 (rpc), 445 (smb)
|-
| [[Data_Transfer/WebDAV|WebDAV]]
| go to solution for restricted networks
| [✓]
| ✓
| ✓
| up to 100GBit/sec
| 80 (http), 443 (https)
|- style="vertical-align:middle;"
| [[Data_Transfer/Graphical_Clients#MobaXterm|MobaXterm]]
| Graphical User Interface (GUI)
| [[Data_Transfer/Graphical_Clients#MobaXterm|✓]]
| ☓
| ☓
| see sftp
| see sftp
|- style="vertical-align:middle;"
| [[SDS@hd/Access/NFS|NFS]]
| mount for multi-user environments
| ☓
| ☓
| [[SDS@hd/Access/NFS|✓]]
| up to 40 Gbit/sec
| -
|- style="vertical-align:middle;"
| [[Data_Transfer/SSHFS|SSHFS]]
| mount, needs stable internet connection
| ☓
| [[Data_Transfer/SSHFS#MacOS_&_Linux|✓]]
| [[Data_Transfer/SSHFS#MacOS_&_Linux|✓]]
| see sftp
| see sftp
|- style="vertical-align:middle;"
| [[Data_Transfer/SFTP|SFTP]]
| interactive shell, better usability when used together with Rclone
| [[Data_Transfer/SFTP#Windows|✓]]
| [[Data_Transfer/SFTP#MacOS_&_Linux|✓]]
| [[Data_Transfer/SFTP#MacOS_&_Linux|✓]]
| up to 40 Gbit/sec
| 22 (ssh)
|}
Table 1: SDS@hd transfer routes

=== Access from a bwHPC Cluster ===

'''bwUniCluster''' 
You can't mount to your $HOME directory but you can create a mount under $TMPDIR by following the instructions for [[Data_Transfer/Rclone#Usage_Rclone_Mount | Rclone mount]]. It is advised to wait a couple of seconds (<code>sleep 5</code>) before trying to use the mounted directory.

'''bwForCluster Helix''' 
You can directly access your storage space under ''/mnt/sds-hd/'' on all login and compute nodes.

'''bwForCluster BinAC 2''' 
You can directly access your storage space under ''/mnt/sds-hd/'' on all login and compute nodes. The prerequisites are:
* The SV responsible has enabled the SV on BinAC 2 once by writing to [mailto:sds-hd-support@urz.uni-heidelberg.de sds-hd-support@urz.uni-heidelberg.de]
* You have a valid kerberos ticket, which can be fetched with <code>kinit <userID></code>

'''Other''' 
You can mount your SDS@hd SV on the cluster yourself by using [[Data_Transfer/Rclone | Rclone]] with the [[Data_Transfer/Rclone#Usage_Rclone_Mount | Rclone mount]] command. As transfer protocol you can use WebDAV or sftp. For a full overview please have a look at [[Data_Transfer/All_Data_Transfer_Routes | All Data Transfer Routes]].

=== Access via Webbrowser (read-only) ===

Visit [https://lsdf02-webdav.urz.uni-heidelberg.de/ lsdf02-webdav.urz.uni-heidelberg.de] and login with your SDS@hd username and service password. Here you can get an overview of the data in your "Speichervorhaben" and download single files. To be able to do more, like moving data, uploading new files, or downloading complete folders, a suitable client is needed as described above.

== Best Practices ==

* '''Managing access rights'''
** When the SV is mounted on Windows, access rights can be adjusted in the file/folder properties menu. (Users from Heidelberg without Windows can use the [https://www.urz.uni-heidelberg.de/de/service-katalog/desktop-und-arbeitsplatz/windows-terminalserver Windows terminal server]).
** bwForCluster Helix users can change the access rights via Helix by using [[Workspace#Regular_Unix_Permissions | unix permissions]] or [[Workspace#ACLs:_Access_Control_Lists | access control lists]]. On non native mounts ACL changes won't work.
** Additionally to the already mentioned options, the SVV can take over ownership of files by opening a [[Data_Transfer/SFTP|SFTP]] shell and running <code>chown -R <userID> <path/to/folder></code> .

* '''Multiuser environment''' → Use [[SDS@hd/Access/NFS|NFS]]

== Troubleshooting ==

Maintance windows and known issues are communicated via the email list or the News section at the SDS@hd start page in the wiki.

* '''Issue:''' The credentials aren't accepted
*: Check if your credentials work in general by trying the access via webbrowser: [https://lsdf02-webdav.urz.uni-heidelberg.de/].
*: If this doesn't work:
*:: → [[Registration/Login/Username| Check your Username]]
*:: → [[Registration/bwForCluster/Helix#Troubleshooting_with_the_Help_of_bwServices | Troubleshooting with bwServices]]
* '''Other Issue'''
** If available, follow the troubleshooting guide of your specific connection method.
** Make sure to not use LAN and WLAN at the same time to prevent connection problems.
** If you have an institutional account, make sure to be a fully active member of your institution.
If these suggestions didn't help, write to the [mailto:sds-hd-support@urz.uni-heidelberg.de support]. Provide the following information:
* Your operating system
* Does the access via webbrowser work?
*: If yes, provide us with detailed information on how you tried to access your SV (used access method, username, ...).

File:Data transfer diagram simple.jpg

2026-08-14T08:58:07Z

H Schumacher: H Schumacher uploaded a new version of File:Data transfer diagram simple.jpg

== Summary ==
created with Miro

Registration/bwForCluster/Helix

2026-08-13T18:46:42Z

H Schumacher: Added info about "Fetching account failed"

__FORCETOC__

== Prerequisites ==

Steps A + B in the [[Registration/bwForCluster | registration process]] must be finished.

== Registration Steps ==

# Visit the bwForCluster Helix '''[https://bwservices.uni-heidelberg.de registration page bwServices]'''.
#: [[File:bwServices_chooseHomeOrganisation.png|right|300px|thumb|Select your home organization]]
# '''Select your home organization''' from the list on the main page and click Proceed/Fortfahren. You will be directed to the ''Identity Provider'' of your home organization.
# Enter the '''username and password''' of your home organization (usually these credentials are also used for other services like email) and click Login/Einloggen.
# When you log in to bwServices for the first time, an overview will appear, with the account information that your home institution submits to the system. Please verify that all data is valid and then click ''Continue/Weiter''.
# You will be redirected back to the bwServices main page. It shows all the statewide services you have access to. Click '''Register/Registrieren''' under the field labeled ''bwForCluster Helix''.
#: [[File:BwIDM-reg.png|center|frame|Register for Helix]]
# bwForCluster Helix uses a '''2-factor authentication''' (2FA) mechanism to increase security.
#: If you have never registered a 2FA token on bwIDM, the following error message will appear:
#: [[File:Bwidm-3-red.png|center|600px|thumb|Second factor missing.]]
#: Use this '''[https://bwservices.uni-heidelberg.de/user/twofa.xhtml link]''' or select '''My Tokens''' in the main menu.
#: To register a new token, please follow these '''[[Registration/2FA|instructions]]'''.
#: Please complete this step before continuing.
# Read the Terms of Use / Nutzungsbedingungen, place a check mark next to ''I have read and accepted the terms of use'' and click ''Register/Registrieren''.
# '''Set a service password''' for bwForCluster Helix and click ''Save/Speichern''.
#: Be sure to use a secure password that is different from any other passwords you currently use or have used on other systems.
[[File:BwIDM-passwd.png|center|600px|thumb|Set service password]]
{|style="background:#deffee; width:70%; margin: auto;"
|style="padding:5px; background:#cef2e0; text-align:left"|
[[Image:Attention.svg|center|25px]]
|style="padding:5px; background:#cef2e0; text-align:left"|
The service password is your private password for accessing the bwForCluster Helix. Don't ever share it with anyone!
|}

== Registration Website bwServices ==
The registration website '''[https://bwservices.uni-heidelberg.de bwServices]''' offers the following functionality:
* Register for the service.
* Set or change your service password.
* Review your registration details under "Registry Info". This includes your account status (active/lost_access).
* bwServices triggers an update of your user information upon login.
* Registering an [[Registration/SSH | SSH key]]. It can be used for login to ''bwForCluster Helix''.
* Creating tokens for logins that need 2-factor authentication ([[Registration/2FA |2FA]]) like ''bwForCluster Helix'' and ''bwVisu''.
* De-register from the service.

=== Troubleshooting with the Help of bwServices ===
When encountering connection issues with a service, the bwServices website is a good starting point for troubleshooting:
{| style="border: 2px solid #FFD28A; background-color: #ffe0af; padding: 10px; margin-bottom: 1em; width: 100%;"
|-
| '''⚠ Login to bwServices first.''' This triggers an update of your user information within some minutes and solves the connection issue in the following cases:
* User data changed: entitlement, SV/RV status, SV/RV membership status, role or email address changed or uni-id was (re)activated
* Synchronization issues of the service
|}

* <ins>'''Issue:</ins> Expired contract''' or inactive student id:
** No access to state services like the bwHPC clusters or SDS@hd is possible.
** You can see that this is the issue when the login to bwServices via your identity provider is not possible.
* <ins>'''Issue:</ins> Entitlement / RV missing:''' Prerequisites like entitlement ownership or RV membership aren't met (anymore).
** You can see the concrete issue and advice on how to move forward when looking at the registry info of the service on the bwServices website.
** When the issue is solved, the account status changes from LOST_ACCESS to ACTIVE.
* <ins>'''Issue:</ins> Service Password:''' When trying to connect to the service, the service password is not accepted.
** Set a [[#Setting_a_New_Service_Password | new password]].
* <ins>'''Issue:</ins> OTP:''' In the case of login issues with a service that uses 2FA, you can check if you're OTP is working by going in the menu to ''Index'' -> ''My Tokens''.
** If you're token is not working, you might have had too many failed login attempts. You can request a reset of the error counter from the support.
** If the token needs to be reset, please follow the instructions for a [[Helix/Login#Troubleshooting | Lost Token]].
* <ins>'''Issue:</ins> Fetching Account failed:''' This issue can be detected by looking in bwServices -> Registration Details of the Service -> More Details ... -> More Properties -> "Fetching account failed" message instead of "Fetching account success". This means that there is a synchronisation issue in our system that probably causes issues for multiple users. Please report it to the support!

=== Setting a New Service Password ===

At any time, you can set a new service password via [https://bwservices.uni-heidelberg.de/ bwServices] by carrying out the following steps:
# Visit [https://bwservices.uni-heidelberg.de/ bwServices] and select your home organization.
# Authenticate yourself via your home-organizational user id / username and your home-organizational password.
# Find your service and select '''Set Service Password'''.
# Set new service password, repeat it and click the '''Save''' button. Use a secure password that is different from any other passwords you currently use or have used on other systems.
# The page answers e.g. "password has been changed".

File:Sds-hd-registration new.png

2026-08-12T15:15:19Z

H Schumacher: