Workspace

From bwHPC Wiki
Jump to navigation Jump to search

Workspace tools provide temporary scratch space so calles workspaces for your calculation on a central file storage. They are meant to keep data for a limited time – but usually longer than the time of a single job run. It is not meant for permanent storage, hence data in workspaces is not backed up and may be lost in case of problems on the storage system. Please copy/move important results to $HOME or some disks outside the cluster.

Create workspace

To create a workspace you need to state name of your workspace and lifetime in days. A maximum value for lifetime and a maximum number of renewals is defined on each cluster. Execution of:

  $ ws_allocate blah 30

e.g. returns:

  Workspace created. Duration is 720 hours. 
  Further extensions available: 3
  /work/workspace/scratch/username-blah-0

For more information read the program's help, i.e. $ ws_allocate -h.


List all your workspaces

To list all your workspaces, execute:

  $ ws_list

which will return:

  • Workspace ID
  • Workspace location
  • available extensions
  • creation date and remaining time


Find workspace location

Workspace location/path can be prompted for any workspace ID using ws_find, in case of workspace blah:

  $ ws_find blah

returns the one-liner:

  /work/workspace/scratch/username-blah-0


Extend lifetime of your workspace

Any workspace's lifetime can be only extended a cluster-specific number of times. There several commands to extend workspace lifetime

  1. $ ws_extend blah 40
    which extends workspace ID blah by 40 days from now,
  2. $ ws_extend blah
    which extends workspace ID blah by the number days used previously
  3. $ ws_allocate -x blah 40
    which extends workspace ID blah by 40 days from now.


Sharing Workspace Data within your Workgroup

Data in workspaces can be shared with colleagues. Making workspaces world readable/writable using standard unix access rights with chmod is strongly discouraged -- it is too course-grained, allowing all-or-nothing access to your University's peers. It is therefore recommended to use ACLs (Access Control List).

Best practices with respect to ACL usage:

  • Take into account that ACL take precedence over standard unix access rights
  • Use a single set of rules at the level of a workspace
  • Make the entire workspace either read-only or read-write for individual co-workers
  • Optional: Make the entire workspace read-only for your Rechenvorhaben, e.g. for large input data
  • If a more granular set of rules is necessary, consider using additional workspaces
  • The owner of a workspace is responsible for its content and management
  • If setting ACLs for individual co-workers, set ACLs for your user first, otherwise you won't be able to access the files of your co-workers!

Please note that ls (List directory contents) shows ACLs on directories and files only when run as ls -l as in long format, as "plus" sign after the standard unix access rights.

Examples with regard to "my_workspace":

Command Action
getfacl $(ws_find my_workspace) List access rights on the workspace named "my_workspace"
setfacl -Rm u:fr_xy1001:rX,d:u:fr_xy1001:rX $(ws_find my_workspace) Grant user "fr_xy1001" read-only access to the workspace named "my_workspace"
setfacl -Rm u:fr_me0000:rwX,d:u:fr_me0000:rwX $(ws_find my_workspace)

setfacl -Rm u:fr_xy1001:rwX,d:u:fr_xy1001:rwX $(ws_find my_workspace)

Grant user your user "fr_me0000" and "fr_xy1001" read and write access to the workspace named "my_workspace" (see explanation above)
setfacl -Rm g:bw16e001:rX,d:g:bw16e001:rX $(ws_find my_workspace) Grant group (Rechenvorhaben) "bw16e001" read-only access to the workspace named "my_workspace"
setfacl -Rb $(ws_find my_workspace) Remove all ACL rights. Standard Unix access rights apply again.

Delete a workspace

  $ ws_release blah # Manually erase your workspace blah