SUNScholar/Import on the new server

Back to Export & Import

Copy the exports tarball archive to the new server

Copy the tarball (exports.tgz) of items exported on the old machine to the home folder on the new machine. Try the following command to do the copy from one Ubuntu machine to another Ubuntu machine.

Open a terminal on the machine where the items were exported and type:

scp dspace@%old-host%:/home/exports.tgz dspace@%new-host%:/home/ 

Replace %old-host% with host name of your old server. Replace %new-host% with host name of your new machine. It is assumed that the "dspace" user account exists on the new machine.

If the above method does not work, then do the transfer manually using a portable disk or any other method that works for you. You could even try to use WinSCP.

Create the communities and collections on the new server

Now manually create the same communties and collections that were on the old machine, on your new machine. Take note of the collection id's created on the new machine and check them against the old collections id's on the old machine using a spreadsheet, for example:

old-collection-id = new-collection-id

Print a copy of the spreadsheet to use for the following procedures.

Also save a list of submitter email addresses (ePersons) per new collection. You might think of defining one submitter email address (ePerson) to use for the imports only.

Create the folders

Login to the new server and become the root user as follows:

sudo -i

Make an imports folder as follows:

mkdir /home/imports

Make a script folder as follows:

mkdir /root/scripts

Extract the exports tarball archive on the new server

Run the following command to untar the tarball contents into the new server's /home/exports folder.

tar -C /home -xzvf /home/exports.tgz

Check that the /home/exports folder contains the items exported from the old server by typing the following:

ls /home/exports

A list of folders should be displayed that represents your exported collections by collection id. If they are not there then check up on what went wrong with the copy and extraction of the exports tarball archive !

Create the import script

We create a symlink to hold the "imports" folder.

cd /home

ln -s exports imports

Open the import script for editing as follows:

nano /root/scripts/collection-import

Copy and paste the following into the editor:

#!/bin/bash

# Setup the configs.
. config

# Check for an exports folder
if [ ! -d $IMPORT ]; then
    clear
    echo "Have you extracted the collections to the correct folder."
    exit 1
fi

# Go thru the collections
for i in `cat collections` ; do
    echo $i > /tmp/map
    ECID=`cat /tmp/map | awk -F',' '{ print $1 }'` 
    ICID=`cat /tmp/map | awk -F',' '{ print $2 }'`
    echo "Importing old collection:$ECID to new collection:$ICID" >> $IMPORT/import.log
    if [ ! -e $IMPORT/$ECID/$ECID-to-$ICID-items-map-list.csv ] ; then
        $DSHOME/bin/dspace import -a -e $PERSON -s $IMPORT/$ECID -c $HANDLE/$ICID -m $IMPORT/$ECID/$ECID-to-$ICID-items-map-list.csv
        else
	echo "$IMPORT/$ECID collection already imported" >> $IMPORT/import.log
        clear
    fi
done

---

NANO Editor Help

CTL+O	= Save the file and then press Enter
CTL+X	= Exit "nano"
CTL+K	= Delete line
CTL+U	= Undelete line
CTL+W	= Search for %%string%%
CTL+\	= Search for %%string%% and replace with $$string$$
CTL+C	= Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)

---

Please note:

1. Check the DC metadata fields in the destination server and ensure that any customisation from the source server is also updated on the destination server.
2. The "-w" adds the item to the collections workflow. This is for movement of assets not migration.

Now make the script executeable as follows:

chmod 0775 /root/scripts/collection-import

Create import config file

Make a config file as follows:

nano /root/scripts/config

Copy and paste the following into editor.

DSHOME="$HOME"
HANDLE="123456789"
IMPORT="$HOME/imports"
PERSON="submitter@myrepo.ac.za"

Modify the above to suit your new site.

---

NANO Editor Help

CTL+O	= Save the file and then press Enter
CTL+X	= Exit "nano"
CTL+K	= Delete line
CTL+U	= Undelete line
CTL+W	= Search for %%string%%
CTL+\	= Search for %%string%% and replace with $$string$$
CTL+C	= Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)

---

Create import collections file

Make collections file as follows:

nano /root/scripts/collections

Copy and paste the following into the editor.

12,34
34,43
23,56
56,78
21,99

Change the above to reflect the collection handle ID's you want to import using the spreadsheet that maps new collection ID's to old collection ID's. The first item per line in the file is the exported collection id and the second item after the comma is the new import collection id.

---

NANO Editor Help

CTL+O	= Save the file and then press Enter
CTL+X	= Exit "nano"
CTL+K	= Delete line
CTL+U	= Undelete line
CTL+W	= Search for %%string%%
CTL+\	= Search for %%string%% and replace with $$string$$
CTL+C	= Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)

---

Do the import

Run the script as follows:

/root/scripts/collection-import

To watch the log file as the imports happen, type the following after opening another terminal:

tail -f /home/imports/import.log

Test the exports for correctness if imports fail

The imports will fail if the exports are not correct. Use the following to setup a script to check the exports for correctness on the new server. This script checks most things but not all bitstream types only PDF files.

Create export check script

Open the export check script for editing as follows:

nano /root/scripts/collection-check

Copy and paste the following to open editor:

#!/bin/bash

# Setup the configs.
. config

# Clean out the old logs.
rm /root/test.log

# Go thru the collections
for i in `cat collections` ; do
    echo $i > /tmp/map
    ECID=`cat /tmp/map | awk -F',' '{ print $1 }'` 
    ICID=`cat /tmp/map | awk -F',' '{ print $2 }'`
    echo "Checking items in collection:$ECID"
    LIST=`ls $IMPORT/$ECID`
    
    for i in $LIST ; do
	ITM=$i
	cd  $IMPORT/$ECID/$ITM
	if [ ! -e dublin_core.xml ] ; then
	echo "Item: $ITM, No dublin core." >> /root/test.log
	fi
	if [ ! -e handle ] ; then
	echo "Item: $ITM, No handle." >> /root/test.log
	fi
	if [ ! -e contents ] ; then
	echo "Item: $ITM, No contents file." >> /root/test.log
	fi
	if [ ! -e license.txt ] ; then
	echo "Item: $ITM, No license." >> /root/test.log
	fi
	ls $IMPORT/$ECID/$ITM | grep pdf > /tmp/pdf.log
	if [ $? -ne 0 ] ; then
	echo "Item: $ITM, No PDF file." >> /root/test.log
	fi
    done
    cat /root/test.log | mail -s "Exported collections test log" $PERSON
    echo "Check complete and report emailed to dspace admin user"
done

---

NANO Editor Help

CTL+O	= Save the file and then press Enter
CTL+X	= Exit "nano"
CTL+K	= Delete line
CTL+U	= Undelete line
CTL+W	= Search for %%string%%
CTL+\	= Search for %%string%% and replace with $$string$$
CTL+C	= Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)

---

Now make the script executeable as follows:

chmod 0775 /root/scripts/collection-check

Do the exports check

Type the following to do the check:

/root/scripts/collection-check

Now check your email, you should have one with details of the test log.

If there are errors, fix them on the old server. Delete the exports on the old server and do a new export after everything is fixed. Bascially start the export and import over again. At least you now have the scripts to do it over again.