SUNScholar/Import on the new server

Back to Export & Import

Copy the exports tarball archive to the new server
Copy the tarball (exports.tgz) of items exported on the old machine to the home folder on the new machine. Try the following command to do the copy from one Ubuntu machine to another Ubuntu machine.

Open a terminal on the machine where the items were exported and type: scp dspace@%old-host%:/home/exports.tgz dspace@%new-host%:/home/ Replace %old-host% with host name of your old server. Replace %new-host% with host name of your new machine. It is assumed that the "dspace" user account exists on the new machine.

If the above method does not work, then do the transfer manually using a portable disk or any other method that works for you. You could even try to use WinSCP.

Create the communities and collections on the new server
Now manually create the same communties and collections that were on the old machine, on your new machine. Take note of the collection id's created on the new machine and check them against the old collections id's on the old machine using a spreadsheet, for example: old-collection-id = new-collection-id Print a copy of the spreadsheet to use for the following procedures.

Also save a list of submitter email addresses (ePersons) per new collection. You might think of defining one submitter email address (ePerson) to use for the imports only.

Create the folders
Login to the new server and become the root user as follows: sudo -i Make an imports folder as follows: mkdir /home/imports Make a script folder as follows: mkdir /root/scripts

Extract the exports tarball archive on the new server
Run the following command to untar the tarball contents into the new server's /home/exports folder. tar -C /home -xzvf /home/exports.tgz Check that the /home/exports folder contains the items exported from the old server by typing the following: ls /home/exports A list of folders should be displayed that represents your exported collections by collection id. If they are not there then check up on what went wrong with the copy and extraction of the exports tarball archive !

Create the import script
We create a symlink to hold the "imports" folder. cd /home

ln -s exports imports Open the import script for editing as follows: nano /root/scripts/collection-import Copy and paste the following into the editor:
 * 1) !/bin/bash

. config
 * 1) Setup the configs.

if [ ! -d $IMPORT ]; then clear echo "Have you extracted the collections to the correct folder." exit 1 fi
 * 1) Check for an exports folder

for i in `cat collections` ; do   echo $i > /tmp/map ECID=`cat /tmp/map | awk -F',' '{ print $1 }'` ICID=`cat /tmp/map | awk -F',' '{ print $2 }'` echo "Importing old collection:$ECID to new collection:$ICID" >> $IMPORT/import.log if [ ! -e $IMPORT/$ECID/$ECID-to-$ICID-items-map-list.csv ] ; then $DSHOME/bin/dspace import -a -e $PERSON -s $IMPORT/$ECID -c $HANDLE/$ICID -m $IMPORT/$ECID/$ECID-to-$ICID-items-map-list.csv else echo "$IMPORT/$ECID collection already imported" >> $IMPORT/import.log clear fi done
 * 1) Go thru the collections

'''Please note: '''
 * 1) Check the DC metadata fields in the destination server and ensure that any customisation from the source server is also updated on the destination server.
 * 2) The "-w" adds the item to the collections workflow. This is for movement of assets not migration.

Now make the script executeable as follows: chmod 0775 /root/scripts/collection-import

Create import config file
Make a config file as follows: nano /root/scripts/config Copy and paste the following into editor. DSHOME="$HOME" HANDLE="123456789" IMPORT="$HOME/imports" PERSON="submitter@myrepo.ac.za" Modify the above to suit your new site.

Create import collections file
Make collections file as follows: nano /root/scripts/collections Copy and paste the following into the editor. 12,34 34,43 23,56 56,78 21,99 Change the above to reflect the collection handle ID's you want to import using the spreadsheet that maps new collection ID's to old collection ID's. The first item per line in the file is the exported collection id and the second item after the comma is the new import collection id.

Do the import
Run the script as follows: /root/scripts/collection-import

To watch the log file as the imports happen, type the following after opening another terminal: tail -f /home/imports/import.log

Test the exports for correctness if imports fail
The imports will fail if the exports are not correct. Use the following to setup a script to check the exports for correctness on the new server. This script checks most things but not all bitstream types only PDF files.

Create export check script
Open the export check script for editing as follows: nano /root/scripts/collection-check Copy and paste the following to open editor:
 * 1) !/bin/bash

. config
 * 1) Setup the configs.

rm /root/test.log
 * 1) Clean out the old logs.

for i in `cat collections` ; do   echo $i > /tmp/map ECID=`cat /tmp/map | awk -F',' '{ print $1 }'` ICID=`cat /tmp/map | awk -F',' '{ print $2 }'` echo "Checking items in collection:$ECID" LIST=`ls $IMPORT/$ECID` for i in $LIST ; do	ITM=$i cd $IMPORT/$ECID/$ITM if [ ! -e dublin_core.xml ] ; then echo "Item: $ITM, No dublin core." >> /root/test.log fi if [ ! -e handle ] ; then echo "Item: $ITM, No handle." >> /root/test.log fi if [ ! -e contents ] ; then echo "Item: $ITM, No contents file." >> /root/test.log fi if [ ! -e license.txt ] ; then echo "Item: $ITM, No license." >> /root/test.log fi ls $IMPORT/$ECID/$ITM | grep pdf > /tmp/pdf.log if [ $? -ne 0 ] ; then echo "Item: $ITM, No PDF file." >> /root/test.log fi done cat /root/test.log | mail -s "Exported collections test log" $PERSON echo "Check complete and report emailed to dspace admin user" done
 * 1) Go thru the collections

Now make the script executeable as follows: chmod 0775 /root/scripts/collection-check

Do the exports check
Type the following to do the check: /root/scripts/collection-check Now check your email, you should have one with details of the test log.

If there are errors, fix them on the old server. Delete the exports on the old server and do a new export after everything is fixed. Bascially start the export and import over again. At least you now have the scripts to do it over again.