Workspaces/Advanced Features/ws allocate: Difference between revisions

From bwHPC Wiki
Jump to navigation Jump to search
(Created page with "= ws_allocate - Create and Extend Workspaces = {| class="wikitable" |- !style="width:40%" | Works on cluster !style="width:10%" | bwUC 3.0 !style="width:10%" | BinAC2 !style="width:10%" | Helix !style="width:10%" | JUSTUS 2 !style="width:10%" | NEMO2 |- |<tt>ws_allocate -x -u</tt> (extend other user's workspace) | style="text-align:center;" | | style="text-align:center;" | | style="text-align:center;" | | style="text-align:center;" | |style="background-color:#90EE9...")
 
mNo edit summary
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
= ws_allocate - Create and Extend Workspaces =
= ws_allocate - Create and Extend Workspaces =


Create new workspaces or extend existing ones. This is your primary command for managing workspace lifetime.
{| class="wikitable"
|-
!style="width:40%" | Works on cluster
!style="width:10%" | bwUC 3.0
!style="width:10%" | BinAC2
!style="width:10%" | Helix
!style="width:10%" | JUSTUS 2
!style="width:10%" | NEMO2
|-
|<tt>ws_allocate -x -u</tt> (extend other user's workspace)
| style="text-align:center;" |
| style="text-align:center;" |
| style="text-align:center;" |
| style="text-align:center;" |
|style="background-color:#90EE90; text-align:center;" | ✓
|}


== Basic Usage ==
== Basic Usage ==


$ ws_allocate myWs 30 # Create workspace for 30 days
Execution of:

$ ws_allocate myWs 30


Returns:
e.g. returns:
Workspace created. Duration is 720 hours.
Workspace created. Duration is 720 hours.
Line 30: Line 13:
/work/workspace/scratch/username-myWs-0
/work/workspace/scratch/username-myWs-0


'''Capture path in variable:'''
The command returns the path to the new directory, which can be captured in a variable:


$ SCRDIR=$(ws_allocate myWs 10)
$ WORKSPACE=$(ws_allocate myWs 30)
$ cd "$WORKSPACE"


'''Important:''' Creating a workspace a second time with the same command is a no-operation - it always returns the same path. This makes it safe and encouraged to use such a line in batch jobs which are part of a series of jobs working on the same data, no matter if the job was running before or not.
'''Important:''' Running the same command again is safe - it returns the existing workspace path. This makes it ideal for batch job scripts.


== All Options and When to Use Them ==
== Common Options ==

'''Group workspaces (for teams):'''

$ ws_allocate -g myWs 30 # Group-readable
$ ws_allocate -G projectgroup myWs 30 # Group-writable (recommended for collaboration)

'''Specific filesystem (if available):'''

$ ws_allocate -F ffuc myWs 30 # bwUniCluster 3.0: Flash filesystem

'''Extend existing workspace:'''

$ ws_allocate -x myWs 30 # Extend by 30 days
$ ws_allocate -x -u alice myWs 30 # Extend Alice's group workspace

'''Custom reminders:'''

$ ws_allocate -r 7 myWs 30 # Reminder 7 days before expiry
$ ws_allocate -r 7 -x myWs 0 # Update reminder only (no extension)

== All Options ==


{| class="wikitable"
{| class="wikitable"
Line 45: Line 50:
|-
|-
|<tt>-F <filesystem></tt>
|<tt>-F <filesystem></tt>
|Create on specific filesystem
|Specify the filesystem where the workspace should be created
|'''Optional''' - Most clusters have only one default filesystem. Use only when you need specific storage characteristics (speed, capacity) or to balance load across multiple filesystems. List available locations with <tt>ws_list -l</tt> or <tt>ws_find -l</tt>. See [[Workspaces/Advanced_Features/Filesystems|Filesystems guide]].
|Only if your cluster has multiple filesystems. Use <tt>ws_list -l</tt> to check availability. See [[Workspaces/Advanced_Features/Filesystems|Filesystems guide]].
|-
|-
|<tt>-g</tt>
|<tt>-g</tt>
|Create a group-readable workspace
|Group-readable workspace
|'''Recommended''' when working in a team and others need to read your data. The workspace will be visible to group members with <tt>ws_list -g</tt>
|When team members need to read your data
|-
|-
|<tt>-G <groupname></tt>
|<tt>-G <groupname></tt>
|Create a group-writable workspace with sticky bit
|Group-writable workspace
|'''Recommended''' for collaborative work where team members need to write data. Ensures all created files belong to the group. Can be set as default in <tt>~/.ws_user.conf</tt>
|For collaborative work. Set default in <tt>~/.ws_user.conf</tt>
|-
|-
|<tt>-m <mailaddress></tt>
|<tt>-m <email></tt>
|Set email address for reminders
|Custom email for reminders
|Only to override identity provider email
|'''Optional''' - Email addresses come from your identity provider. Only use this option to override with a different address. Can be set as default in <tt>~/.ws_user.conf</tt>
|-
|-
|<tt>-r <days></tt>
|<tt>-r <days></tt>
|Set reminder to be sent n days before expiration
|Reminder n days before expiration
|To customize reminder timing (default is automatic)
|'''Optional''' - Email reminders are sent automatically. Use this only to customize when the reminder starts (e.g., 3, 5, or 7 days before). Can be set as default in <tt>~/.ws_user.conf</tt>
|-
|-
|<tt>-x</tt>
|<tt>-x</tt>
|Extend an existing workspace
|Extend existing workspace
|Use when you need more time. Each extension consumes one of the available extensions
|When you need more time
|-
|-
|<tt>-u <username></tt>
|<tt>-u <username></tt>
|Used with <tt>-x</tt> to extend another user's workspace
|Extend another user's workspace
|Use when a group member is absent and their shared workspace needs extension. Requires group write access <tt>-G</tt>
|When group member is unavailable (requires <tt>-G</tt>)
|-
|-
|<tt>-c <comment></tt>
|<tt>-c <comment></tt>
|Add a comment to the workspace
|Add workspace comment
|Document workspace purpose
|Use to document the purpose of the workspace for yourself and collaborators
|-
|-
|<tt>-d <duration></tt>
|<tt>-d <duration></tt>
|Duration in days (alternative to positional argument)
|Duration in days
|Alternative to positional argument
|Use if you prefer explicit option syntax: <tt>ws_allocate -n myWs -d 30</tt>
|-
|-
|<tt>-n <name></tt>
|<tt>-n <name></tt>
|Workspace name (alternative to positional argument)
|Workspace name
|Alternative to positional argument
|Use if you prefer explicit option syntax: <tt>ws_allocate -n myWs -d 30</tt>
|}
|}


== Duration Settings ==
== Duration Settings ==


If you do not specify a lifetime, a default lifetime will be used (see the [[Workspaces/Advanced_Features/Quotas|Cluster-Specific Workspace Limits]]). The maximum lifetime may be limited by the operations team. If you specify a longer lifetime, it will be capped to the maximum, and you will see a message that it was changed.
If you don't specify duration, the system default is used. Maximum lifetime varies by cluster - see [[Workspaces/Advanced_Features/Quotas|Cluster-Specific Workspace Limits]].


If you request more than the maximum, it will be capped automatically.
For more information read the program's help: <tt>$ ws_allocate -h</tt> or <tt>man ws_allocate</tt>


== Using Workspaces in Batch Jobs ==
== Using Workspaces in Batch Jobs ==


'''Recommended approach:''' Create your workspace manually before submitting jobs, then reference it in your job scripts using <tt>ws_find</tt>.
'''Recommended approach:''' Create workspace manually, then use <tt>ws_find</tt> in job scripts.


'''(1) Create workspace once (on login node):'''
'''(1) Create workspace (on login node):'''


$ ws_allocate myProject 60
$ ws_allocate myProject 60


'''(2) Use in job scripts with ws_find:'''
'''(2) Use in job script:'''


<pre>
<pre>
Line 108: Line 113:
# Find existing workspace
# Find existing workspace
WORKSPACE=$(ws_find myProject)
WORKSPACE=$(ws_find myProject)
cd "$WORKSPACE"


# Your computation
# Change to workspace
cd $WORKSPACE

# Your computation here
./my_program --input input.dat --output results.dat
./my_program --input input.dat --output results.dat
</pre>
</pre>


'''Warning:''' Avoid using <tt>ws_allocate</tt> directly in job scripts that run frequently. While <tt>ws_allocate</tt> is safe to call multiple times on the same workspace name (it returns the existing workspace), you should not create too many workspaces unnecessarily. Create workspaces manually when needed, then use <tt>ws_find</tt> in your job scripts to locate them.
'''Important:''' Don't create workspaces in job scripts that run frequently. Create them manually when needed, then locate them with <tt>ws_find</tt>.

See also: [[Workspaces/Advanced_Features/ws_find|ws_find]]


For more information: <tt>ws_allocate -h</tt> or <tt>man ws_allocate</tt>
See also: [[Workspaces/Advanced_Features/ws_find|ws_find]] for finding workspace paths in scripts.

Latest revision as of 16:54, 2 December 2025

ws_allocate - Create and Extend Workspaces

Create new workspaces or extend existing ones. This is your primary command for managing workspace lifetime.

Basic Usage

  $ ws_allocate myWs 30                    # Create workspace for 30 days

Returns:

  Workspace created. Duration is 720 hours. 
  Further extensions available: 3
  /work/workspace/scratch/username-myWs-0

Capture path in variable:

  $ WORKSPACE=$(ws_allocate myWs 30)
  $ cd "$WORKSPACE"

Important: Running the same command again is safe - it returns the existing workspace path. This makes it ideal for batch job scripts.

Common Options

Group workspaces (for teams):

  $ ws_allocate -g myWs 30                 # Group-readable
  $ ws_allocate -G projectgroup myWs 30    # Group-writable (recommended for collaboration)

Specific filesystem (if available):

  $ ws_allocate -F ffuc myWs 30            # bwUniCluster 3.0: Flash filesystem

Extend existing workspace:

  $ ws_allocate -x myWs 30                 # Extend by 30 days
  $ ws_allocate -x -u alice myWs 30        # Extend Alice's group workspace

Custom reminders:

  $ ws_allocate -r 7 myWs 30               # Reminder 7 days before expiry
  $ ws_allocate -r 7 -x myWs 0             # Update reminder only (no extension)

All Options

Option Description When to Use
-F <filesystem> Create on specific filesystem Only if your cluster has multiple filesystems. Use ws_list -l to check availability. See Filesystems guide.
-g Group-readable workspace When team members need to read your data
-G <groupname> Group-writable workspace For collaborative work. Set default in ~/.ws_user.conf
-m <email> Custom email for reminders Only to override identity provider email
-r <days> Reminder n days before expiration To customize reminder timing (default is automatic)
-x Extend existing workspace When you need more time
-u <username> Extend another user's workspace When group member is unavailable (requires -G)
-c <comment> Add workspace comment Document workspace purpose
-d <duration> Duration in days Alternative to positional argument
-n <name> Workspace name Alternative to positional argument

Duration Settings

If you don't specify duration, the system default is used. Maximum lifetime varies by cluster - see Cluster-Specific Workspace Limits.

If you request more than the maximum, it will be capped automatically.

Using Workspaces in Batch Jobs

Recommended approach: Create workspace manually, then use ws_find in job scripts.

(1) Create workspace (on login node):

  $ ws_allocate myProject 60

(2) Use in job script:

#!/bin/bash
#SBATCH --job-name=my_job
#SBATCH --time=24:00:00

# Find existing workspace
WORKSPACE=$(ws_find myProject)
cd "$WORKSPACE"

# Your computation
./my_program --input input.dat --output results.dat

Important: Don't create workspaces in job scripts that run frequently. Create them manually when needed, then locate them with ws_find.

See also: ws_find

For more information: ws_allocate -h or man ws_allocate