BwUniCluster3.0/Data Migration Guide

Summary of changes

bwUniCluster 3.0 is located on the North Campus in order to meet the requirements of energy-efficient and environmentally more friendly HPC operation via the hot water cooling available there. bwUniCluster 3.0 has a new file system.

Hardware

Policy

Software

Data Migration

How to move data from bwUniCluster 2.0 to bwUniCluster 3.0

You have to check the quotas, new quotas are more stringent than old ones. Users should pay attention to their data lifecycle management.

Assisted data migration

To facilitate the transfer of data between the old and new HOME directories and workspaces, we provide a script that guides you through the copy process or even automates the transfer:

migrate_data_uc2_uc3.sh -h

In order to mitigate the effects of the quota changes, the script first performs a quota check. If a quota check detects that the storage capacity or the number of inodes has been exceeded, the program terminates with an error message.

If the quota check was without objection, the data migration command is displayed. For the fearless, the -x flag can even be used to initiate the copy process itself.

The script can automate the data transfer to the new HOME directory. If you intend to also transfer data resident in workspaces, the script can automate this, too. However, the target workspaces on the new system first have to be setup manually!

Manual migration of HOME

1. Check the Quota

Show user quota of the old $HOME:
$ lfs quota -uh $USER /pfs/data5
Show user quota of the new $HOME:
$ lfs quota -uh $USER /pfs/data6
For the new file system, the limit for capacity and inodes must be higher than the capacity and the number of inodes used in the old file system in order to avoid I/O errors during data transfer. Pay attention to the respective limit, used, and files column of the outputs.

2. Cleanup

If the capacity limit or the maximum number of inodes is exceeded, now is the right time to clean up.
Either delete data in the source directory before the rsync command or use additional --exclude statements during rsync.
Hint:
If the inode limit is exceeded, you should, for example, delete all existing Python virtual environments, which often contain a massive number of small files.

3. Migrate the data

# Rsync des home:
OLDHOME=${HOME//ka_/}
OLDHOME=${OLDHOME/\/home\/ka//home/kit}
OLDHOME=${OLDHOME/\/home\///pfs/data5/home/}
# Alternative: OLDHOME=$(echo $HOME | sed -e "s/ka_//g" -e "s|^/home/ka|/home/kit|" -e "s|^/home|/pfs/data5/home|")
rsync -x --numeric-ids -S -rlptoD -H -A --exclude='/.*' ${OLDHOME}/ ${HOME}/
# Kurzbeschreibung der rsync Optionen:
# -x don't cross filesystem boundaries
# --numeric-ids don't map uid/gid values by user/group name
# -S turn sequences of nulls into sparse blocks
# -r recurse into directories
# -l copy symlinks as symlinks
# -p preserve permissions
# -t preserve modification times
# Hinweis: -g bewusst weggelassen, weil sich Gruppe bei OE-Wechsel geändert haben kann
# -o preserve owner
# -D preserve device files and preserve special files
# -H preserve hard links: brauchen wir das?
# -A preserve ACLs: brauchen und wollen wir das?
# --exclude='/.*' Dateien und Verzeichnisse, die im alten Home liegen und mit . starten sind ausgeschlossen, siehe
# https://www.baeldung.com/linux/recursive-copy-skip-hidden-files#:~:text=Using%20the%20rsync%20Command&text=It%20accepts%20two%20valuable%20options,pattern%20to%20the%20%E2%80%93exclude%20option.
# weitere evtl nützliche Optionen:
# -v increase verbosity
# -n perform a trial run with no changes made



# Generell:
# Die Daten sollten auf Compute-Knoten (interaktiv oder in Batch Jobs) übertragen werden, um
# Parallelität über Clients zu erreichen und die Überlastung der Login-Knoten zu vermeiden.

Manual migration of Workspaces

# User Quotas der alten Workspaces anzeigen:
lfs quota -uh $USER /pfs/work7
# User Quotas der neuen Workspaces anzeigen:
lfs quota -uh $USER /pfs/work9

# Rsync der workspaces:
OLDUSER=${USER//ka_/}
# Pfade zu alten Workspaces anzeigen:
ls -d /pfs/work7/workspace/scratch/${OLDUSER}-*
# Für jeden zu kopierenden Workspace neuen Workspace anlegen mit ws_allocate und Daten mit rsync kopieren:
rsync -x --numeric-ids -S -rlptoD -H -A /Pfad/zu/altem/Workspace/mit/abschließendem/Slash/ /Pfad/zu/neuem/Workspace/mit/abschließendem/Slash/

Migration of Software and Settings

We explicitly exclude all dot files and dot directories (.bashrc, .config/, ...)