BwUniCluster3.0/Data Migration Guide

Summary of changes

bwUniCluster 3.0 is located on the North Campus in order to meet the requirements of energy-efficient and environmentally more friendly HPC operation via the hot water cooling available there. bwUniCluster 3.0 has a new file system.

Data Migration

How to move data from bwUniCluster 2.0 to bwUniCluster 3.0

You have to check the quotas, new quotas are more stringent than old ones. Users should pay attention to their data lifecycle management.

HOME directory

Check the Quotas

# Show user quotas of the old $HOME:
lfs quota -uh $USER /pfs/data5
# Show user quotas of the new $HOME:
lfs quota -uh $USER /pfs/data6

1. Check
For the new file system, the limit for capacity and inodes must be higher than the capacity and the number of inodes used in the old file system in order to avoid I/O errors during data transfer.

(limit column in each case) (used column) (files column)

2. Cleanup
If the capacity limit or the maximum number of inodes is exceeded, now is the right time to clean up. If the capacity limit or the maximum number of inodes is exceeded, now is the right time to clean up.
Either delete data in the source directory before the rsync command or use additional --exclude statements during rsync.
Hint: If the inode limit is exceeded, you should, for example, delete all existing Python virtual environments, which often contain a massive number of small files.

1. Aufräumen während des Umzugs der Daten:
2. Entweder vor dem rsync-Kommando Daten im Quellverzeichnis löschen oder beim rsync weitere
3. exclude-Statements verwenden.

1. Rsync des home:

OLDHOME=${HOME//ka_/} OLDHOME=${OLDHOME/\/home\/ka//home/kit} OLDHOME=${OLDHOME/\/home\///pfs/data5/home/}

1. Alternative: OLDHOME=$(echo $HOME | sed -e "s/ka_//g" -e "s|^/home/ka|/home/kit|" -e "s|^/home|/pfs/data5/home|")

rsync -x --numeric-ids -S -rlptoD -H -A --exclude='/.*' ${OLDHOME}/ ${HOME}/

1. Kurzbeschreibung der rsync Optionen:
2. -x don't cross filesystem boundaries
3. --numeric-ids don't map uid/gid values by user/group name
4. -S turn sequences of nulls into sparse blocks
5. -r recurse into directories
6. -l copy symlinks as symlinks
7. -p preserve permissions
8. -t preserve modification times
9. Hinweis: -g bewusst weggelassen, weil sich Gruppe bei OE-Wechsel geändert haben kann
10. -o preserve owner
11. -D preserve device files and preserve special files
12. -H preserve hard links: brauchen wir das?
13. -A preserve ACLs: brauchen und wollen wir das?
14. --exclude='/.*' Dateien und Verzeichnisse, die im alten Home liegen und mit . starten sind ausgeschlossen, siehe
15. https://www.baeldung.com/linux/recursive-copy-skip-hidden-files#:~:text=Using%20the%20rsync%20Command&text=It%20accepts%20two%20valuable%20options,pattern%20to%20the%20%E2%80%93exclude%20option.
16. weitere evtl nützliche Optionen:
17. -v increase verbosity
18. -n perform a trial run with no changes made

1. Rsync der workspaces:

OLDUSER=${USER//ka_/}

1. Pfade zu alten Workspaces anzeigen:

ls -d /pfs/work7/workspace/scratch/${OLDUSER}-*

1. Für jeden zu kopierenden Workspace neuen Workspace anlegen mit ws_allocate und Daten mit rsync kopieren:

rsync -x --numeric-ids -S -rlptoD -H -A /Pfad/zu/altem/Workspace/mit/abschließendem/Slash/ /Pfad/zu/neuem/Workspace/mit/abschließendem/Slash/

1. Generell:
2. Die Daten sollten auf Compute-Knoten (interaktiv oder in Batch Jobs) übertragen werden, um
3. Parallelität über Clients zu erreichen und die Überlastung der Login-Knoten zu vermeiden.

Workspaces

# User Quotas der alten Workspaces anzeigen:
lfs quota -uh $USER /pfs/work7
# User Quotas der neuen Workspaces anzeigen:
lfs quota -uh $USER /pfs/work9

Migration of Software and Settings

We explicitly exclude all dot files and dot directories (.bashrc, .config/, ...)