M Janczyk: M Janczyk moved page Workspaces/Advanced Features/Best Practices to NEMO2/Workspaces/Advanced Features/Best Practices

2026-03-02T19:39:29Z

M Janczyk moved page Workspaces/Advanced Features/Best Practices to NEMO2/Workspaces/Advanced Features/Best Practices

← Older revision	Revision as of 21:39, 2 March 2026
(No difference)

M Janczyk: Created page with "= Best Practices and Recommendations = == For All Users == # '''Set up ~/.ws_user.conf''' - Configure default reminder timing, duration, and groupname to avoid typing them repeatedly (see Reminders & Configuration) # '''Email reminders are automatic''' - Notifications are sent automatically using your identity provider email; only use `-r` to customize reminder timing if needed # '''Custom email only if needed''' - Only..."

2025-12-02T13:41:28Z

Created page with "= Best Practices and Recommendations = == For All Users == # '''Set up ~/.ws_user.conf''' - Configure default reminder timing, duration, and groupname to avoid typing them repeatedly (see Reminders & Configuration) # '''Email reminders are automatic''' - Notifications are sent automatically using your identity provider email; only use <tt>-r</tt> to customize reminder timing if needed # '''Custom email only if needed''' - Only..."

New page

= Best Practices and Recommendations =

== For All Users ==

# '''Set up ~/.ws_user.conf''' - Configure default reminder timing, duration, and groupname to avoid typing them repeatedly (see [[Workspaces/Advanced_Features/Reminders|Reminders & Configuration]])
# '''Email reminders are automatic''' - Notifications are sent automatically using your identity provider email; only use <tt>-r</tt> to customize reminder timing if needed
# '''Custom email only if needed''' - Only use <tt>-m</tt> option to override the email address from your identity provider
# '''Use ws_register''' - Create symbolic links to your workspaces in a convenient directory (see [[Workspaces/Advanced_Features/ws_register|ws_register]])
# '''Create workspaces manually''' - Create workspaces on the login node before submitting jobs, then use <tt>ws_find</tt> in your job scripts (see [[Workspaces/Advanced_Features/ws_allocate#Using_Workspaces_in_Batch_Jobs|Using Workspaces in Batch Jobs]])
# '''Track your workspaces''' - Regularly run <tt>ws_list -R</tt> to see which workspaces will expire soon
# '''Backup important data''' - Workspaces are temporary and not backed up - copy results to appropriate permanent storage (check your cluster/site policies for backup locations)
# '''Clean up regularly''' - Release workspaces you no longer need to keep filesystems organized

== For Short-term Jobs (hours to days) ==

# Use default or shorter durations
# Consider using a single workspace for a series of related jobs
# Use <tt>ws_find</tt> in job scripts to locate the workspace (see [[Workspaces/Advanced_Features/ws_allocate#Using_Workspaces_in_Batch_Jobs|Using Workspaces in Batch Jobs]])
# Copy results to permanent storage when jobs complete
# Release workspace when no longer needed (see [[Workspaces/Release|Release Workspace]])

== For Long-term Campaigns (weeks to months) ==

# Request maximum allowed duration (see [[Workspaces/Advanced_Features/Quotas|Cluster-Specific Workspace Limits]])
# Email reminders are sent automatically; optionally customize reminder timing with <tt>-r</tt> option
# Use <tt>ws_list -R</tt> regularly to monitor remaining time (see [[Workspaces/List|List Your Workspaces]])
# Plan data archival to appropriate permanent storage before expiration (check cluster/site policies)

== For Collaborative Work ==

# Use <tt>ws_allocate -G groupname</tt> for shared write access (see [[Workspaces/Groups#Create_Group_Workspace|Create Group Workspace]])
# Set <tt>groupname</tt> in <tt>~/.ws_user.conf</tt> if you always work with the same group (see [[Workspaces/Advanced_Features/Reminders|Reminders & Configuration]])
# Use <tt>ws_allocate -g</tt> for read-only sharing within group
# Use <tt>ws_list -g</tt> to see all group workspaces (see [[Workspaces/Groups#List_Group_Workspaces|List Group Workspaces]])
# Team members can extend group workspaces (see [[Workspaces/Groups#Extend_Group_Workspace|Extend Group Workspace]])
# Take over reminder responsibility when colleague is unavailable (see [[Workspaces/Groups#Manage_Reminders_for_Group_Workspace|Manage Reminders]])
# Document the workspace location for your team members
# For advanced sharing scenarios (ACL-based, ws_share), see the [[Workspaces/Advanced_Features/Sharing|Sharing guide]]

== For Managing Multiple Filesystems ==

# '''Note:''' Most clusters have only one default filesystem - the <tt>-F</tt> option is rarely needed
# Use <tt>ws_list -l</tt> first to check if multiple filesystems are available on your cluster
# Use <tt>-F</tt> option only if you need specific filesystem for performance or capacity needs (see [[Workspaces/Advanced_Features/Filesystems|Filesystems guide]])
# '''bwUniCluster 3.0 filesystems:'''
#* '''Default Lustre filesystem:''' Standard workspace location, best for large files and sequential I/O
#* '''Flash filesystem (ffuc):''' SSD-based storage for KIT/HoreKa users, shared between bwUniCluster 3.0 and HoreKa
#* Use flash filesystem for workloads with many small files, random I/O, AI/ML training, or compilation
#* Balance load: use <tt>-F ffuc</tt> when appropriate to reduce load on default filesystem
# '''General guidelines:'''
#* Flash-based filesystems (SSD/NVMe): Use for many small files, low-latency requirements, random I/O
#* Standard Lustre/parallel filesystems: Best for large files and sequential I/O patterns

== For Different Data Types ==

# '''Large sequential I/O:''' Use standard workspace filesystem (Lustre best for very large files, Weka excellent for both large and small)
# '''Many small files or random access:''' Use flash-based workspace filesystem like Weka (NEMO2) or bwUniCluster ffuc, or stage to <tt>$TMPDIR</tt>
# '''Data read multiple times on single node:''' Copy to <tt>$TMPDIR</tt> at job start for best performance
# '''Temporary data for single node:''' Always use <tt>$TMPDIR</tt>, not workspaces
# '''Multi-node temporary data:''' Use workspaces (not suitable for <tt>$TMPDIR</tt>)
# '''AI/ML training data:''' Use Weka (NEMO2) or flash filesystems for best performance, or stage to <tt>$TMPDIR</tt> for repeated access
# '''Compilation/build directories:''' Use flash-based filesystems (Weka, ffuc) or <tt>$TMPDIR</tt> for better performance

== For Quota Management ==

* Use <tt>ws_release --delete-data</tt> for immediate deletion (see [[Workspaces/Advanced_Features/ws_release#Immediate_Deletion_.28Free_Quota_Instantly.29|Immediate Deletion]])
* For clusters without <tt>--delete-data</tt> option, use [[Workspaces/Advanced_Features/ws_release#Using_rm_for_Manual_Deletion|manual deletion method]]
* Remember: released workspaces may still count toward quota during grace period (~1 hour)

NEMO2/Workspaces/Advanced Features/Best Practices - Revision history

M Janczyk: M Janczyk moved page Workspaces/Advanced Features/Best Practices to NEMO2/Workspaces/Advanced Features/Best Practices