BwUniCluster3.0/Data Migration Guide

Summary of changes

bwUniCluster 3.0 is located on the North Campus in order to meet the requirements of energy-efficient and environmentally more friendly HPC operation via the hot water cooling available there. bwUniCluster 3.0 has a new file system.

Data Migration

How to move data from bwUniCluster 2.0 to bwUniCluster 3.0

You have to check the quotas, new quotas are more stringent than old ones. Users should pay attention to their data lifecycle management.

HOME directory

1. Check the Quota

# Show user quota of the old $HOME:
lfs quota -uh $USER /pfs/data5
# Show user quota of the new $HOME:
lfs quota -uh $USER /pfs/data6

For the new file system, the limit for capacity and inodes must be higher than the capacity and the number of inodes used in the old file system in order to avoid I/O errors during data transfer.

(limit column in each case) (used column) (files column)

2. Cleanup

If the capacity limit or the maximum number of inodes is exceeded, now is the right time to clean up. If the capacity limit or the maximum number of inodes is exceeded, now is the right time to clean up.
Either delete data in the source directory before the rsync command or use additional --exclude statements during rsync.
Hint: If the inode limit is exceeded, you should, for example, delete all existing Python virtual environments, which often contain a massive number of small files. Furthermore, the .vscode-server directory

3. Migrate the data

# Rsync des home:
OLDHOME=${HOME//ka_/}
OLDHOME=${OLDHOME/\/home\/ka//home/kit}
OLDHOME=${OLDHOME/\/home\///pfs/data5/home/}
# Alternative: OLDHOME=$(echo $HOME | sed -e "s/ka_//g" -e "s|^/home/ka|/home/kit|" -e "s|^/home|/pfs/data5/home|")
rsync -x --numeric-ids -S -rlptoD -H -A --exclude='/.*' ${OLDHOME}/ ${HOME}/
# Kurzbeschreibung der rsync Optionen:
# -x don't cross filesystem boundaries
# --numeric-ids don't map uid/gid values by user/group name
# -S turn sequences of nulls into sparse blocks
# -r recurse into directories
# -l copy symlinks as symlinks
# -p preserve permissions
# -t preserve modification times
# Hinweis: -g bewusst weggelassen, weil sich Gruppe bei OE-Wechsel geändert haben kann
# -o preserve owner
# -D preserve device files and preserve special files
# -H preserve hard links: brauchen wir das?
# -A preserve ACLs: brauchen und wollen wir das?
# --exclude='/.*' Dateien und Verzeichnisse, die im alten Home liegen und mit . starten sind ausgeschlossen, siehe
# https://www.baeldung.com/linux/recursive-copy-skip-hidden-files#:~:text=Using%20the%20rsync%20Command&text=It%20accepts%20two%20valuable%20options,pattern%20to%20the%20%E2%80%93exclude%20option.
# weitere evtl nützliche Optionen:
# -v increase verbosity
# -n perform a trial run with no changes made

# Rsync der workspaces:
OLDUSER=${USER//ka_/}
# Pfade zu alten Workspaces anzeigen:
ls -d /pfs/work7/workspace/scratch/${OLDUSER}-*
# Für jeden zu kopierenden Workspace neuen Workspace anlegen mit ws_allocate und Daten mit rsync kopieren:
rsync -x --numeric-ids -S -rlptoD -H -A /Pfad/zu/altem/Workspace/mit/abschließendem/Slash/ /Pfad/zu/neuem/Workspace/mit/abschließendem/Slash/

# Generell:
# Die Daten sollten auf Compute-Knoten (interaktiv oder in Batch Jobs) übertragen werden, um
# Parallelität über Clients zu erreichen und die Überlastung der Login-Knoten zu vermeiden.

Workspaces

# User Quotas der alten Workspaces anzeigen:
lfs quota -uh $USER /pfs/work7
# User Quotas der neuen Workspaces anzeigen:
lfs quota -uh $USER /pfs/work9

Migration of Software and Settings

We explicitly exclude all dot files and dot directories (.bashrc, .config/, ...)