Workspace: Difference between revisions
K Siegmund (talk | contribs) |
K Siegmund (talk | contribs) |
||
Line 51: | Line 51: | ||
<br> |
<br> |
||
== Setting Permissions for Sharing Workspace Data |
== Setting Permissions for Sharing Workspace Data == |
||
The examples will assume you want to change the directory in $DIR. If you want to share a workspace, DIR could be set with <code>DIR=$(ws_find my_workspace)</code> |
|||
Options are: |
|||
Data in workspaces can be shared with colleagues. Making workspaces world readable/writable using standard unix access rights with <tt>chmod</tt> is strongly discouraged for bwUniCluster, as the groups are defined too coarse-grained, allowing access for too many people. They may work for you on the bwForClusters, where you and your co-workers of a research group share a common bwXXXXX unix group. |
|||
* regular unix permissions |
|||
Making workspaces world readable/writable using standard unix access rights with <tt>chmod</tt> is only feasible if you are in a research group and you and your co-workers share a common ("bwXXXXX") unix group. It is strongly discouraged to make files readable or even writable to everyone or to large common groups. |
|||
⚫ | ACLs have the additional advantage that you can set a "default" ACL for a directory, (with a <tt>-d</tt> flag or a <tt>d:</tt> prefix) which will cause all newly created files to inherit the ACLs from the directory. Regular unix permissions only have limited support (only group ownership, not access rights) for this via the suid bit. |
||
{| class="wikitable" |
|||
⚫ | |||
!style="width:45%" | Command |
|||
!style="width:55%" | Action |
|||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
|- |
|||
|} |
|||
* "ACL"s access crontrol lists |
|||
It is therefore recommended to use ACLs (Access Control List) if they are available on the cluster. |
|||
⚫ | ACLs allow a much more detailed distribution of permissions but are a bit more complicated and not visible in detail via "ls". They have the additional advantage that you can set a "default" ACL for a directory, (with a <tt>-d</tt> flag or a <tt>d:</tt> prefix) which will cause all newly created files to inherit the ACLs from the directory. Regular unix permissions only have limited support (only group ownership, not access rights) for this via the suid bit. |
||
Best practices with respect to ACL usage: |
Best practices with respect to ACL usage: |
||
# Take into account that ACL take precedence over standard unix access rights |
|||
# The owner of a workspace is responsible for its content and management |
|||
Please note that <tt>ls</tt> (List directory contents) shows ACLs on directories and files only when run as <tt>ls -l</tt> as in long format, as "plus" sign after the standard unix access rights. |
Please note that <tt>ls</tt> (List directory contents) shows ACLs on directories and files only when run as <tt>ls -l</tt> as in long format, as "plus" sign after the standard unix access rights. |
||
Line 71: | Line 88: | ||
!style="width:55%" | Action |
!style="width:55%" | Action |
||
|- |
|- |
||
|<tt>getfacl $ |
|<tt>getfacl $DIR</tt> |
||
|List access rights on the workspace named "my_workspace" |
|List access rights on the workspace named "my_workspace" |
||
|- |
|- |
||
|<tt>setfacl -Rm u:fr_xy1:rX,d:u:fr_xy1:rX $ |
|<tt>setfacl -Rm u:fr_xy1:rX,d:u:fr_xy1:rX $DIR</tt> |
||
|Grant user "fr_xy1" read-only access to the workspace named "my_workspace" |
|Grant user "fr_xy1" read-only access to the workspace named "my_workspace" |
||
|- |
|- |
||
|<tt>setfacl -Rm u:fr_me0000:rwX,d:u:fr_me0000:rwX $ |
|<tt>setfacl -Rm u:fr_me0000:rwX,d:u:fr_me0000:rwX $DIR</tt> |
||
<tt>setfacl -Rm u:fr_xy1:rwX,d:u:fr_xy1:rwX $(ws_find my_workspace)</tt> |
<tt>setfacl -Rm u:fr_xy1:rwX,d:u:fr_xy1:rwX $(ws_find my_workspace)</tt> |
||
|Grant your own user "fr_me0000" and "fr_xy1" inheritable read and write access to the workspace named "my_workspace", so you can also read/write files put into the workspace by a coworker |
|Grant your own user "fr_me0000" and "fr_xy1" inheritable read and write access to the workspace named "my_workspace", so you can also read/write files put into the workspace by a coworker |
||
|- |
|- |
||
|<tt>setfacl -Rm g:bw16e001:rX,d:g:bw16e001:rX $ |
|<tt>setfacl -Rm g:bw16e001:rX,d:g:bw16e001:rX $DIR</tt> |
||
|Grant group (Rechenvorhaben) "bw16e001" read-only access to the workspace named "my_workspace" |
|Grant group (Rechenvorhaben) "bw16e001" read-only access to the workspace named "my_workspace" |
||
|- |
|- |
||
|<tt>setfacl -Rb $ |
|<tt>setfacl -Rb $DIR</tt> |
||
|Remove all ACL rights. Standard Unix access rights apply again. |
|Remove all ACL rights. Standard Unix access rights apply again. |
||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
⚫ | |||
|} |
|} |
||
Revision as of 12:11, 5 March 2025
Workspace tools provide temporary scratch space so calles workspaces for your calculation on a central file storage. They are meant to keep data for a limited time – but usually longer than the time of a single job run.
No Backup
Workspaces are not meant for permanent storage, hence data in workspaces is not backed up and may be lost in case of problems on the storage system. Please copy/move important results to $HOME or some disks outside the cluster.
Create workspace
To create a workspace you need to state name of your workspace and lifetime in days. A maximum value for lifetime and a maximum number of renewals is defined on each cluster. Execution of:
$ ws_allocate blah 30
e.g. returns:
Workspace created. Duration is 720 hours. Further extensions available: 3 /work/workspace/scratch/username-blah-0
For more information read the program's help, i.e. $ ws_allocate -h.
List all your workspaces
To list all your workspaces, execute:
$ ws_list
which will return:
- Workspace ID
- Workspace location
- available extensions
- creation date and remaining time
Find workspace location
Workspace location/path can be prompted for any workspace ID using ws_find, in case of workspace blah:
$ ws_find blah
returns the one-liner:
/work/workspace/scratch/username-blah-0
Extend lifetime of your workspace
Any workspace's lifetime can be only extended a cluster-specific number of times. There several commands to extend workspace lifetime
$ ws_extend blah 40
which extends workspace ID blah by 40 days from now,$ ws_extend blah
which extends workspace ID blah by the number days used previously$ ws_allocate -x blah 40
which extends workspace ID blah by 40 days from now.
Setting Permissions for Sharing Workspace Data
The examples will assume you want to change the directory in $DIR. If you want to share a workspace, DIR could be set with DIR=$(ws_find my_workspace)
Options are:
- regular unix permissions
Making workspaces world readable/writable using standard unix access rights with chmod is only feasible if you are in a research group and you and your co-workers share a common ("bwXXXXX") unix group. It is strongly discouraged to make files readable or even writable to everyone or to large common groups.
Command | Action |
---|---|
chgrp -R bw16e001 $DIR
chmod -R g+rX $DIR |
Set group ownership and grant read access to group for files in workspace via unix rights (has to be re-done if files are added) |
chgrp -R bw16e001 $(ws_find my_workspace)
chmod -R g+rswX $(ws_find my_workspace) |
Set group ownership and grant read/write access to group for files in workspace via unix rights (has to be re-done if files are added). Group will be inherited by new files, but rights for the group will have to be re-set with chmod for every new file |
- "ACL"s access crontrol lists
ACLs allow a much more detailed distribution of permissions but are a bit more complicated and not visible in detail via "ls". They have the additional advantage that you can set a "default" ACL for a directory, (with a -d flag or a d: prefix) which will cause all newly created files to inherit the ACLs from the directory. Regular unix permissions only have limited support (only group ownership, not access rights) for this via the suid bit.
Best practices with respect to ACL usage:
- Take into account that ACL take precedence over standard unix access rights
- The owner of a workspace is responsible for its content and management
Please note that ls (List directory contents) shows ACLs on directories and files only when run as ls -l as in long format, as "plus" sign after the standard unix access rights.
Examples with regard to "my_workspace":
Command | Action |
---|---|
getfacl $DIR | List access rights on the workspace named "my_workspace" |
setfacl -Rm u:fr_xy1:rX,d:u:fr_xy1:rX $DIR | Grant user "fr_xy1" read-only access to the workspace named "my_workspace" |
setfacl -Rm u:fr_me0000:rwX,d:u:fr_me0000:rwX $DIR
setfacl -Rm u:fr_xy1:rwX,d:u:fr_xy1:rwX $(ws_find my_workspace) |
Grant your own user "fr_me0000" and "fr_xy1" inheritable read and write access to the workspace named "my_workspace", so you can also read/write files put into the workspace by a coworker |
setfacl -Rm g:bw16e001:rX,d:g:bw16e001:rX $DIR | Grant group (Rechenvorhaben) "bw16e001" read-only access to the workspace named "my_workspace" |
setfacl -Rb $DIR | Remove all ACL rights. Standard Unix access rights apply again. |
Delete a workspace
$ ws_release blah # Manually erase your workspace blah