Workspace

Workspace tools provide temporary scratch space so called workspaces for your calculation on a central file storage. They are meant to keep data for a limited time. This can be a couple of days or several months.

No Backup

Workspaces are not meant for permanent storage, hence data in workspaces is usually not backed up and may be lost in case of problems on the storage system. Please copy/move important results to $HOME or a storage space outside the cluster.

Create workspace

To create a workspace you need to state the name of your workspace and the lifetime in days:

ws_allocate <name> <lifetime>

The maximum value for lifetime and a maximum number of renewals is specific to each cluster.

Example: The command

ws_allocate mySpace 30

returns:

Workspace created. Duration is 720 hours. 
Further extensions available: 3
/work/workspace/scratch/username-mySpace-0

For more information read the program's help, i.e. ws_allocate -h.

List all your workspaces

To list all your workspaces, execute:

ws_list

which will return:

• Workspace ID
• Workspace location
• available extensions
• creation date and remaining time

To adjust the output, you can check the help: ws_list -h. The option -R for example sorts the workspaces by remaining time.

To list expired workspaces, see Restore an Expired Workspace.

Find workspace location

The workspace location/path can be prompted for any workspace ID using ws_find, in case of workspace mySpace:

ws_find mySpace

returns the one-liner:

/work/workspace/scratch/username-mySpace-0

You can combine the command with the cd command to diretly move there:

cd $(ws_find mySpace)

Extend lifetime of your workspace

Any workspace's lifetime can only be extended a cluster-specific number of times. There are several commands to extend workspace lifetime

1. ws_extend mySpace 40
   # or 
   ws_allocate -x mySpace 40
   

   which extends workspace ID mySpace by 40 days from now.

2. ws_extend mySpace
   

   which extends workspace ID mySpace by the number of days used previously.

Setting Permissions for Sharing Files

The examples will assume you want to change the directory $DIR. If you want to share a workspace, DIR could be set with DIR=$(ws_find my_workspace)

Workspace Tools

• ws_share allows to share an existing workspace with other users by providing read access:

  # Share/unshare workspace
  ws_share <share / unshare> <workspacename> <username> [username2]
  # Remove read access from all users that the workspace was shared with
  ws_share unshare-all <workspacename> 
  # List all users that were granted read access
  ws_share list <workspacename>
  

• Grant read and write access to a group (RV, SV, ...) when creating a new workspace. (Only works with newer versions of the workspace tools.)

  ws_allocate -G <groupname> <workspacename> <duration>
  

  e.g. for the group bw16e001 and for the workspace commonfiles for 30 days:

  ws_allocate -G bw16e001 commonfiles 30
  

  Essentially, this tool sets regular unix rwx permissions for the group plus the "suid" bit on the directory to make the permission inheritable.

Regular Unix Permissions

Making workspaces readable/writable using standard unix access rights with chmod is only feasible if you and your co-workers share a common unix group (for example "bw22a222" or sd22a222).

Do not make files readable or even writable to everyone or to large common groups ("all students").

You can see available groups with the command groups.

Command	Action
chgrp -R bw16e001 "$DIR" chmod -R g+rX "$DIR"	Set group ownership and grant read access to group for files in workspace via unix rights to the group "bw16e001" (has to be re-done if files are added)
chgrp -R bw16e001 "$DIR" chmod -R g+rswX "$DIR"	Set group ownership and grant read/write access to group for files in workspace via unix rights (has to be re-done if files are added). Group will be inherited by new files, but rights for the group will have to be re-set with chmod for every new file

Options used:

• -R: recursive
• g+rwx
  • g: group
  • +: add permissions (- to remove)
  • rwx: read, write, execute

ACLs: Access Control Lists

ACLs allow a much more detailed distribution of permissions but are a bit more complicated and not visible in detail via "ls". They have the additional advantage that you can set a "default" ACL for a directory, (with a -d flag or a d: prefix) which will cause all newly created files to inherit the ACLs from the directory. Regular unix permissions only have limited support (only group ownership, not access rights) for this via the suid bit.

Best practices with respect to ACL usage:

1. Take into account that ACLs take precedence over standard unix access rights
2. The owner of a workspace is responsible for its content and management

Please note that ls (List directory contents) shows ACLs on directories and files only when run as ls -l as in long format, as "plus" sign after the standard unix access rights.

Examples with regard to "my_workspace":

Command	Action
getfacl "$DIR"	List access rights on $DIR
setfacl -Rm user:fr_xy1:rX,default:user:fr_xy1:rX "$DIR"	Grant user "fr_xy1" read-only access to $DIR
setfacl -R -m user:fr_me0000:rwX,default:user:fr_me0000:rwX "$DIR" setfacl -R -m user:fr_xy1:rwX,default:user:fr_xy1:rwX "$DIR"	Grant your own user "fr_me0000" and "fr_xy1" inheritable ("default") read and write access to $DIR, so you can also read/write files put into the workspace by a coworker
setfacl -Rm group:bw16e001:rX,default:group:bw16e001:rX "$DIR"	Grant group (Rechenvorhaben) "bw16e001" read-only access to $DIR
setfacl -Rb "$DIR"	Remove all ACL rights. Standard Unix access rights apply again.

Options used:

• -R: recursive
• -m: modify
• user:username:rwX user: next name is a user; rwX read, write, eXecute (only where execute is set for user)
• default:[user|group]: set the default for user or group for new files or dierctories

Delete a Workspace

# Manually erase your workspace mySpace
ws_release mySpace

Note: workspaces are kept for some time after release. To immediately delete and free space e.g. for quota reasons, delete the files with rm before release.

Newer versions of workspace tools have a --delete-data flag that immediately deletes data. Note that deleted data from workspaces is permanently lost.

Restore an Expired Workspace

For a certain (system-specific) grace time following workspace expiration, a workspace can be restored by performing the following steps:

(1) Display restorable workspaces.

ws_restore -l

(2) Create a new workspace as the target for the restore:

ws_allocate <new_workspace> <lifetime>

(3) Restore:

ws_restore <full_name_of_expired_workspace> <new_workspace>

The expired workspace has to be specified using the full name, including username prefix and timestamp suffix (otherwise, it cannot be uniquely identified). The target workspace, on the other hand, must be given with just its short name as listed by ws_list, without the username prefix.

In most cases: If the workspace is not visible/restorable, it has been permanently deleted and cannot be restored, not even by us. Please always remember, that workspaces are intended solely for temporary work data, and there is no backup of data in the workspaces. Please see the workspace page of your specific cluster in case your cluster does have some kind of backup.

Troubleshooting

• Error: "Disk space exceeded"

  This means that there is not enough space (e.g. in your $HOME) to execute the operation. Please free up space.