BinAC2/Migrate BinAC 1 workspaces to BinAC 2 workspaces
Preamble
Remove unnecessary data
Now is a good possibility getting rid of unnecessary data. Don't migrate data to BinAC 2 that you know you won't ever touch again. Instead backup it if needed and delete it from BinAC 1.
BinAC 2 storage architecture
Please read about the BinAC 2 file systems and understand the difference between project and work file system before migrating any data.
About rsync
You will use rsync for migrating your data from BinAC 1 to BinAC 2. You may want to use the following options:
-a: archive mode; equals -rlptgoD (no -H, -A, -X) -h: output numbers in a human-readable format -x: don't cross filesystem boundaries -z: compress file data during the transfer -P: keep partially transferred files --progress: show progress during transfer
 |
Mind trailing |
About screen and tmux
It is recommended to migrate data in a screen or tmux session, as the migration may take a while.
You can detach from the session by pressing Ctrl+A, D (Ctrl+B, D, for tmux).
You can reattach to the session by running screen -r (tmux attach).
You can cancel the migration by pressing Ctrl+C.
# For screen users: # start a screen session screen -S migratebinacws # start data migration, commands see below # detach from the screen session Ctrl+A, D # reattach to the screen session screen -r migratenemo1ws # cancel migration, if needed Ctrl+C # exit screen session, when done Ctrl+D # For tmux users: # start a tmux session tmux new -s migratebinacws # start data migration, commands see below # detach from the tmux session Ctrl+B, D # reattach to the tmux session tmux attach -t migratenemo1ws # cancel migration, if needed Ctrl+C # exit tmux session, when done Ctrl+D
Migration
All commands assume that you are logged in on BinAC 1 and you also have a working account on BinAC 2.
If you used a directory at /beegfs/work for sharing data with people from the same compute project you can sync this directory to the new project file system at /pfs/10/project/.
Depending on how you want to organize your data in the new project file system we will give some example commands:
This command will copy the content of the shared directory on BinAC 1 into the project directory on BinAC 2:
rsync -ahxzP --progress /beegfs/work/<your shared directory>/* ${USER}@login.binac2.uni-tuebingen.de:/pfs/10/project/<RV acronym>/
This command will copy the shared directory on BinAC 1 into the projet directory on BinAC 2. E.g. it will create a subdirectory /pfs/10/project/<RV acronym>/
rsync -ahxzP --progress /beegfs/work/<your shared directory>/ ${USER}@login.binac2.uni-tuebingen.de:/pfs/10/project/<RV acronym>/<subdir name>
Migrate personal BinAC 1 workspaces
This can be a directory at /beegfs/work or a workspace created by the workspace tools at /beegfs/work/workspace/ws.
If the directory contains data you don't need for computations in the next time, please migrate it to a personal directory in the projects file system: Although project is accessible for every compute project member, you can create directories only you can access. This will save precious space on the fast (but not tinier) BinAC 2 work.
# On BinAC 1
rsync -ahxzP --progress /beegfs/work/workspace/ws/<directory name>/ ${USER}@login.binac2.uni-tuebingen.de:/pfs/10/project/<RV acronym>/<directory name>
# On BinAC 2: Change permissions. Only you can access the directory, nobody else.
chmod 700 /pfs/10/project/<RV acronym>/<directory name>
If the directory contains data you will use for computations it's totally fine to migrate it to BinAC 2 work file system.
Please not that (in contrast to BinAC 1) you cannot create directories yourself in the work file system with mkdir. Everything at /pfs/10/work is a workspace managed by the workspace tools.
For each of your workspaces on BinAC 1 you want to migrate you will have to create a workspace on BinAC 2:
ws_allocate <workspace name> 30
You can now sync the files from BinAC 1 to BinAC 2:
This command will copy the content of the workspace on BinAC 1 into the workspace on BinAC 2:
rsync -ahxzP --progress /beegfs/work/workspace/ws/<directory name>/* ${USER}@login.binac2.uni-tuebingen.de:/pfs/10/work/<workspace name>/