BwUniCluster2.0/File System Migration Guide: Difference between revisions
(Created page with "This page describes changes on the file systems between bwUniCluster 1 and bwUniCluster 2.0. = Lustre file systems = With bwUniCluster 2.0 new file systems for $HOME and wor...") |
|||
Line 8: | Line 8: | ||
You should see the same data as before but if this is not true - which is possible in rare cases since the data of expired users has been cleaned up - please contact the hotline or open a ticket. The virtual path to $HOME (/home/ORG/GROUP/USER) is still the same (except for University of Stuttgart users, see below) but the physical path (which can be seen by ''/bin/pwd'') has changed. If one of yours scripts includes a physical path (wich is not recommended) you should modify these scripts. If applications complain about the old physical path (which started with ''/pfs/data1/'' or ''/pfs/data2/'') you should modify or recompile the application. |
You should see the same data as before but if this is not true - which is possible in rare cases since the data of expired users has been cleaned up - please contact the hotline or open a ticket. The virtual path to $HOME (/home/ORG/GROUP/USER) is still the same (except for University of Stuttgart users, see below) but the physical path (which can be seen by ''/bin/pwd'') has changed. If one of yours scripts includes a physical path (wich is not recommended) you should modify these scripts. If applications complain about the old physical path (which started with ''/pfs/data1/'' or ''/pfs/data2/'') you should modify or recompile the application. |
||
== Hints for migration of $WORK data == |
|||
Workspaces should be used instead of $WORK. Advantages of workspaces are a clear deletion policy and the ability to restore data from expired workspaces for few weeks. The old $WORK will be mounted in read-only mode for 4 weeks. If data on $WORK is still needed it has to be migrated by the users. Example for the migration to a workspace: |
|||
<pre> |
|||
# create a new workspace |
|||
$ ws_allocate newwork |
|||
# define environment variable for old WORK |
|||
OLDWORK=$WORK |
|||
# define environment variable for workspace |
|||
WORK=$(ws_find newwork) |
|||
# Copy the data with rsync |
|||
rsync -a -A $OLDWORK/ $WORK/ |
|||
</pre> |
Revision as of 14:41, 13 March 2020
This page describes changes on the file systems between bwUniCluster 1 and bwUniCluster 2.0.
Lustre file systems
With bwUniCluster 2.0 new file systems for $HOME and workspaces have been purchased. The user data of the corresponding old file systems has been copied by KIT/SCC to these new file systems. The content of $WORK has to be migrated manually by the users to workspaces during the next 4 weeks, for details see below.
What has changed on $HOME and workspace directories?
You should see the same data as before but if this is not true - which is possible in rare cases since the data of expired users has been cleaned up - please contact the hotline or open a ticket. The virtual path to $HOME (/home/ORG/GROUP/USER) is still the same (except for University of Stuttgart users, see below) but the physical path (which can be seen by /bin/pwd) has changed. If one of yours scripts includes a physical path (wich is not recommended) you should modify these scripts. If applications complain about the old physical path (which started with /pfs/data1/ or /pfs/data2/) you should modify or recompile the application.
Hints for migration of $WORK data
Workspaces should be used instead of $WORK. Advantages of workspaces are a clear deletion policy and the ability to restore data from expired workspaces for few weeks. The old $WORK will be mounted in read-only mode for 4 weeks. If data on $WORK is still needed it has to be migrated by the users. Example for the migration to a workspace:
# create a new workspace $ ws_allocate newwork # define environment variable for old WORK OLDWORK=$WORK # define environment variable for workspace WORK=$(ws_find newwork) # Copy the data with rsync rsync -a -A $OLDWORK/ $WORK/