Workspaces/Advanced Features/Best Practices

Best Practices and Recommendations

For All Users

1. Set up ~/.ws_user.conf - Configure default reminder timing, duration, and groupname to avoid typing them repeatedly (see Reminders & Configuration)
2. Email reminders are automatic - Notifications are sent automatically using your identity provider email; only use -r to customize reminder timing if needed
3. Custom email only if needed - Only use -m option to override the email address from your identity provider
4. Use ws_register - Create symbolic links to your workspaces in a convenient directory (see ws_register)
5. Create workspaces manually - Create workspaces on the login node before submitting jobs, then use ws_find in your job scripts (see Using Workspaces in Batch Jobs)
6. Track your workspaces - Regularly run ws_list -R to see which workspaces will expire soon
7. Backup important data - Workspaces are temporary and not backed up - copy results to appropriate permanent storage (check your cluster/site policies for backup locations)
8. Clean up regularly - Release workspaces you no longer need to keep filesystems organized

For Short-term Jobs (hours to days)

1. Use default or shorter durations
2. Consider using a single workspace for a series of related jobs
3. Use ws_find in job scripts to locate the workspace (see Using Workspaces in Batch Jobs)
4. Copy results to permanent storage when jobs complete
5. Release workspace when no longer needed (see Release Workspace)

For Long-term Campaigns (weeks to months)

1. Request maximum allowed duration (see Cluster-Specific Workspace Limits)
2. Email reminders are sent automatically; optionally customize reminder timing with -r option
3. Use ws_list -R regularly to monitor remaining time (see List Your Workspaces)
4. Plan data archival to appropriate permanent storage before expiration (check cluster/site policies)

For Collaborative Work

1. Use ws_allocate -G groupname for shared write access (see Create Group Workspace)
2. Set groupname in ~/.ws_user.conf if you always work with the same group (see Reminders & Configuration)
3. Use ws_allocate -g for read-only sharing within group
4. Use ws_list -g to see all group workspaces (see List Group Workspaces)
5. Team members can extend group workspaces (see Extend Group Workspace)
6. Take over reminder responsibility when colleague is unavailable (see Manage Reminders)
7. Document the workspace location for your team members
8. For advanced sharing scenarios (ACL-based, ws_share), see the Sharing guide

For Managing Multiple Filesystems

1. Note: Most clusters have only one default filesystem - the -F option is rarely needed
2. Use ws_list -l first to check if multiple filesystems are available on your cluster
3. Use -F option only if you need specific filesystem for performance or capacity needs (see Filesystems guide)
4. bwUniCluster 3.0 filesystems:
   • Default Lustre filesystem: Standard workspace location, best for large files and sequential I/O
   • Flash filesystem (ffuc): SSD-based storage for KIT/HoreKa users, shared between bwUniCluster 3.0 and HoreKa
   • Use flash filesystem for workloads with many small files, random I/O, AI/ML training, or compilation
   • Balance load: use -F ffuc when appropriate to reduce load on default filesystem
5. General guidelines:
   • Flash-based filesystems (SSD/NVMe): Use for many small files, low-latency requirements, random I/O
   • Standard Lustre/parallel filesystems: Best for large files and sequential I/O patterns

For Different Data Types

1. Large sequential I/O: Use standard workspace filesystem (Lustre best for very large files, Weka excellent for both large and small)
2. Many small files or random access: Use flash-based workspace filesystem like Weka (NEMO2) or bwUniCluster ffuc, or stage to $TMPDIR
3. Data read multiple times on single node: Copy to $TMPDIR at job start for best performance
4. Temporary data for single node: Always use $TMPDIR, not workspaces
5. Multi-node temporary data: Use workspaces (not suitable for $TMPDIR)
6. AI/ML training data: Use Weka (NEMO2) or flash filesystems for best performance, or stage to $TMPDIR for repeated access
7. Compilation/build directories: Use flash-based filesystems (Weka, ffuc) or $TMPDIR for better performance

For Quota Management

• Use ws_release --delete-data for immediate deletion (see Immediate Deletion)
• For clusters without --delete-data option, use manual deletion method
• Remember: released workspaces may still count toward quota during grace period (~1 hour)