Workspace

Workspace tools provide temporary scratch space so calles workspaces for your calculation on a central file storage. They are meant to keep data for a limited time – but usually longer than the time of a single job run.

No Backup

Workspaces are not meant for permanent storage, hence data in workspaces is not backed up and may be lost in case of problems on the storage system. Please copy/move important results to $HOME or some disks outside the cluster.

Create workspace

To create a workspace you need to state name of your workspace and lifetime in days. A maximum value for lifetime and a maximum number of renewals is defined on each cluster. Execution of:

  $ ws_allocate mySpace 30

e.g. returns:

  Workspace created. Duration is 720 hours. 
  Further extensions available: 3
  /work/workspace/scratch/username-mySpace-0

For more information read the program's help, i.e. $ ws_allocate -h.

List all your workspaces

To list all your workspaces, execute:

  $ ws_list

which will return:

• Workspace ID
• Workspace location
• available extensions
• creation date and remaining time

To list expired workspaces, see Restore an Expired Workspace.

Find workspace location

Workspace location/path can be prompted for any workspace ID using ws_find, in case of workspace mySpace:

  $ ws_find mySpace

returns the one-liner:

  /work/workspace/scratch/username-mySpace-0

Extend lifetime of your workspace

Any workspace's lifetime can be only extended a cluster-specific number of times. There several commands to extend workspace lifetime

1. $ ws_extend mySpace 40

   which extends workspace ID mySpace by 40 days from now,

2. $ ws_extend mySpace

   which extends workspace ID mySpace by the number days used previously

3. $ ws_allocate -x mySpace 40

   which extends workspace ID mySpace by 40 days from now.

Setting Permissions for Sharing Files

The examples will assume you want to change the directory in $DIR. If you want to share a workspace, DIR could be set with DIR=$(ws_find my_workspace)

Workspace Tools

• ws_share

ws_share share workspacename username

allows you to grant the user username read access to the workspace.

Newer versions of the workspace tools have sharing options to ws_allocate:

• -G option of ws_allocate

ws_allocate -G groupname workspacename duration

• • groupname: name of the group you want to share with
  • workspacename: what you want to call your workspace
  • duration: how long the workspace is supposed to last in days

Essentially this tool sets regular unix rwx permissions for the group plus the "suid" bit on the directory to make the permission inheritable.

Regular Unix Permissions

Making workspaces world readable/writable using standard unix access rights with chmod is only feasible if you are in a research group and you and your co-workers share a common ("bwXXXXX") unix group.

Do not make files readable or even writable to everyone or to large common groups ("all students").

Command	Action
chgrp -R bw16e001 "$DIR" chmod -R g+rX "$DIR"	Set group ownership and grant read access to group for files in workspace via unix rights to the group "bw16e001" (has to be re-done if files are added)
chgrp -R bw16e001 "$DIR" chmod -R g+rswX "$DIR"	Set group ownership and grant read/write access to group for files in workspace via unix rights (has to be re-done if files are added). Group will be inherited by new files, but rights for the group will have to be re-set with chmod for every new file

Options used:

• -R: recursive
• g+rwx
  • g: group
  • + add permissions (- to remove)
  • rwx: read, write, execute

"ACL"s: Access Crontrol Lists

ACLs allow a much more detailed distribution of permissions but are a bit more complicated and not visible in detail via "ls". They have the additional advantage that you can set a "default" ACL for a directory, (with a -d flag or a d: prefix) which will cause all newly created files to inherit the ACLs from the directory. Regular unix permissions only have limited support (only group ownership, not access rights) for this via the suid bit.

Best practices with respect to ACL usage:

1. Take into account that ACL take precedence over standard unix access rights
2. The owner of a workspace is responsible for its content and management

Please note that ls (List directory contents) shows ACLs on directories and files only when run as ls -l as in long format, as "plus" sign after the standard unix access rights.

Examples with regard to "my_workspace":

Command	Action
getfacl "$DIR"	List access rights on $DIR
setfacl -Rm user:fr_xy1:rX,default:user:fr_xy1:rX "$DIR"	Grant user "fr_xy1" read-only access to $DIR
setfacl -R -m user:fr_me0000:rwX,default:user:fr_me0000:rwX "$DIR" setfacl -R -m user:fr_xy1:rwX,default:user:fr_xy1:rwX "$DIR"	Grant your own user "fr_me0000" and "fr_xy1" inheritable ("default") read and write access to $DIR, so you can also read/write files put into the workspace by a coworker
setfacl -Rm group:bw16e001:rX,default:group:bw16e001:rX "$DIR"	Grant group (Rechenvorhaben) "bw16e001" read-only access to $DIR
setfacl -Rb "$DIR"	Remove all ACL rights. Standard Unix access rights apply again.

Options used:

• -R: recursive
• -m: modify
• user:username:rwX user: next name is a user; rwX read, write, eXecute (only where execute is set for user)
• default:[user|group] set the default for user or group for new files or dierctories

Delete a Workspace

  $ ws_release mySpace # Manually erase your workspace mySpace

Note: workspaces are kept for some time after release. To immediately delete and free space e.g. for quota reasons, delete the files with rm before release.

Newer versions of workspace tools have a --delete-data flag that immediately deletes data. Note that deleted data from workspaces is permanently lost.

Restore an Expired Workspace

For a certain (system-specific) grace time following workspace expiration, a workspace can be restored by performing the following steps:

(1) Display restorable workspaces.

ws_restore -l

(2) Create a new workspace as the target for the restore:

ws_allocate restored 60

(3) Restore:

ws_restore <full_name_of_expired_workspace> restored

The expired workspace has to be specified using the full name, including username prefix and timestamp suffix (otherwise, it cannot be uniquely identified). The target workspace, on the other hand, must be given with just its short name as listed by ws_list, without the username prefix.

If the workspace is no visible/restorable, it has been permanently deleted and cannot be restored, not even by us. Please always remember, that workspaces are intended solely for temporary work data, and there is no backup of data in the workspaces.