Difference between revisions of "SUNScholar/Export and Import Artifacts"

From Libopedia
Jump to navigation Jump to search
Line 3: Line 3:
 
comment5 http://us.startv.com/shows/love daily diet advice http://us.startv.com/news/spooky free web site search engine submit http://us.startv.com/synopsis/birthday-specials best prescription diet drug http://us.startv.com/news/kamya-punjabi-talks-about-maryaada-lekin-kab-tak diet pepsi pilot http://us.startv.com/userf phermine diet http://us.startv.com/gallery/zara wabc radio weight loss sponsor http://us.startv.com/node/7/downloads eygptian diet http://us.startv.com/gallery/neil-deepika-chhote-ustaad-3 deadweight welfare loss http://us.startv.com/wallpapers/humari-devrani-wallpaper free weightloss journal http://us.startv.com/news/vivian-speaks-about-abhay directories for submission http://us.startv.com/wallpapers/masterchef cardio burns fat http://us.startv.com/recaps/masterchef weightloss camps for kids http://us.startv.com/wallpapers/masterchef-india-0 japanese macrobiotic diet http://us.startv.com/gallery/lkp-2 submit your masturbation technique http://us.startv.com/synopsis/ajab diete proteinee
 
comment5 http://us.startv.com/shows/love daily diet advice http://us.startv.com/news/spooky free web site search engine submit http://us.startv.com/synopsis/birthday-specials best prescription diet drug http://us.startv.com/news/kamya-punjabi-talks-about-maryaada-lekin-kab-tak diet pepsi pilot http://us.startv.com/userf phermine diet http://us.startv.com/gallery/zara wabc radio weight loss sponsor http://us.startv.com/node/7/downloads eygptian diet http://us.startv.com/gallery/neil-deepika-chhote-ustaad-3 deadweight welfare loss http://us.startv.com/wallpapers/humari-devrani-wallpaper free weightloss journal http://us.startv.com/news/vivian-speaks-about-abhay directories for submission http://us.startv.com/wallpapers/masterchef cardio burns fat http://us.startv.com/recaps/masterchef weightloss camps for kids http://us.startv.com/wallpapers/masterchef-india-0 japanese macrobiotic diet http://us.startv.com/gallery/lkp-2 submit your masturbation technique http://us.startv.com/synopsis/ajab diete proteinee
  
−
=Import on the new server=
+
comment6 http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewMonth&tx_teksaccext_pi4[month]=092010 healthy low calorie diets http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=2852&tx_teksaccext_pi3[address]= robotics equipment submit your site http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewMonth&tx_teksaccext_pi4[month]=021801&tx_teksaccext_pi4[expandEvents]=true&tx_teksaccext_pi4[category]=12 mike mahler diet http://www.springfieldchamber.com/index.php?id=124&no_cache=1 with the adkins diet http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=emailForm&tx_teksaccext_pi3[uid]=294 submission erotic story http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewMonth&tx_teksaccext_pi4[month]=102010&tx_teksaccext_pi4[expandEvents]=true diet soda phosphorus peyronies syndrome http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewMonth&tx_teksaccext_pi4[month]=021792 acid alkalin diet http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=1517&tx_teksaccext_pi3[address]= blood type diet food lists http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=2920&tx_teksaccext_pi3[address]= does the diet solution work http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=6168&tx_teksaccext_pi3[address]= $153 weight loss pill http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=1110&tx_teksaccext_pi3[address]= diet food raw start http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewdate&tx_teksaccext_pi4[date]=1287464400 com diet healthy loss newnitetrim program weight http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewMonth&tx_teksaccext_pi4[month]=021801&tx_teksaccext_pi4[expandEvents]=true&tx_teksaccext_pi4[category]=5 individual states bmi http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=2174&tx_teksaccext_pi3[address]= quick weight loss centers cost http://www.springfieldchamber.com/no_cache/business_directory/?tx_teksaccext_pi1[mode]=redirWWW&tx_teksaccext_pi1[uid]=5374&tx_teksaccext_pi1[address]= heart medical hospital diet
−
==Copy the exports tarball archive to the new server==
 
−
Copy the tarball (exports.tgz) of items exported on the old machine to the home folder on the new machine. Try the following command to do the copy from one Ubuntu machine to another Ubuntu machine. Open a terminal on the machine where the items were exported and type:
 
−
scp dspace@%old-host%:/home/exports.tgz dspace@%new-host%:/home/  
 
−
Replace %old-host% with host name of your old server. Replace %new-host% with host name of your new machine. It is assumed that the "dspace" user account exists on the new machine.
 
−
 
 
−
If the above method does not work, then do the transfer manually using a portable disk or any other method that works for you. You could even try to use '''WinSCP'''.
 
−
==Create the communities and collections on the new server==
 
−
Now manually create the same communties and collections that were on the old machine, on your new machine. Take note of the collection id's created on the new machine (see the screenshot and instructions above) and check them against the old collections id's on the old machine using a spreadsheet, for example:
 
−
old-collection-id = new-collection-id
 
−
Print a copy of the spreadsheet to use for the following procedures.
 
−
 
 
−
Also save a list of submitter email addresses (ePersons) per new collection. You might think of defining one submitter email address (ePerson) to use for the imports only.
 
−
 
 
−
==Create the folders==
 
−
Login to the new server and become the root user as follows:
 
−
sudo -i
 
−
Make an imports folder as follows:
 
−
mkdir /home/imports
 
−
Make a script folder as follows:
 
−
mkdir /root/scripts
 
−
 
 
−
==Extract the exports tarball archive on the new server==
 
−
Run the following command to untar the tarball contents into the new server's '''/home/exports''' folder.
 
−
tar -C /home -xzvf /home/exports.tgz
 
−
Check that the '''/home/exports''' folder contains the items exported from the old server by typing the following:
 
−
ls /home/exports
 
−
A list of folders should be displayed that represents your exported collections by collection id. If they are not there then check up on what went wrong with the copy and extraction of the exports tarball archive !
 
−
 
 
−
==Create the import script==
 
−
We create a symlink to hold the "imports" folder.
 
−
cd /home
 
−
 
 
−
ln -s exports imports
 
−
Open the import script for editing as follows:
 
−
nano /root/scripts/collection-import
 
−
Copy and paste the following into the editor:
 
−
<pre>
 
−
#!/bin/bash
 
−
 
 
−
# Setup the configs.
 
−
. config
 
−
 
 
−
# Check for an exports folder
 
−
if [ ! -d $IMPORT ]; then
 
−
    clear
 
−
    echo "Have you extracted the collections to the correct folder."
 
−
    exit 1
 
−
fi
 
−
 
 
−
# Go thru the collections
 
−
for i in `cat collections` ; do
 
−
    echo $i > /tmp/map
 
−
    ECID=`cat /tmp/map | awk -F',' '{ print $1 }'`
 
−
    ICID=`cat /tmp/map | awk -F',' '{ print $2 }'`
 
−
    echo "Importing old collection:$ECID to new collection:$ICID" >> $IMPORT/import.log
 
−
    if [ ! -e $IMPORT/$ECID/$ECID-to-$ICID-items-map-list.csv ] ; then
 
−
        $DSHOME/bin/import -a -w -e $PERSON -s $IMPORT/$ECID -c $HANDLE/$ICID -m $IMPORT/$ECID/$ECID-to-$ICID-items-map-list.csv
 
−
        else
 
−
echo "$IMPORT/$ECID collection already imported" >> $IMPORT/import.log
 
−
        clear
 
−
    fi
 
−
done
 
−
</pre>
 
−
 
 
−
'''Please note:<font color="red">
 
−
#Check the DC metadata fields in the destination server and ensure that any customisation from the source server is also updated on the destination server.
 
−
#The -w means add to the collections workflow. This is for movement of assets not migration.
 
−
</font>'''
 
−
 
 
−
Save the file and exit the editor.
 
−
 
 
−
Now make the script executeable as follows:
 
−
chmod 0775 /root/scripts/collection-import
 
−
 
 
−
==Create import config file==
 
−
Make a config file as follows:
 
−
nano /root/scripts/config
 
−
Copy and paste the following into editor.
 
−
<pre>
 
−
HANDLE="123456789"
 
−
IMPORT="/home/imports"
 
−
DSHOME="/home/dspace"
 
−
PERSON="submitter@myrepo.ac.za"
 
−
</pre>
 
−
Modify the above to suit your new site and then save and exit the file.
 
−
 
 
−
==Create import collections file==
 
−
Make collections file as follows:
 
−
nano /root/scripts/collections
 
−
Copy and paste the following into the editor.
 
−
<pre>
 
−
12,34
 
−
34,43
 
−
23,56
 
−
56,78
 
−
21,99
 
−
</pre>
 
−
'''Change the above to reflect the collection handle ID's you want to import using the spreadsheet that maps new collection ID's to old collection ID's. The first item per line in the file is the exported collection id and the second item after the comma is the new import collection id.'''
 
−
 
 
−
Save the file and exit.
 
−
 
 
−
==Do the import==
 
−
Run the script as follows:
 
−
/root/scripts/collection-import
 
−
 
 
−
To watch the log file as the imports happen, type the following after opening another terminal:
 
−
tail -f /home/imports/import.log
 
  
 
=Test the exports for correctness if imports fail=
 
=Test the exports for correctness if imports fail=

Revision as of 22:54, 8 March 2011

hZOHT7 <a href="http://etimddixgjqy.com/">etimddixgjqy</a>, [url=http://lehdsaofbuys.com/]lehdsaofbuys[/url], [link=http://lyszaopiwaoj.com/]lyszaopiwaoj[/link], http://yhitepssatwb.com/

comment5 http://us.startv.com/shows/love daily diet advice http://us.startv.com/news/spooky free web site search engine submit http://us.startv.com/synopsis/birthday-specials best prescription diet drug http://us.startv.com/news/kamya-punjabi-talks-about-maryaada-lekin-kab-tak diet pepsi pilot http://us.startv.com/userf phermine diet http://us.startv.com/gallery/zara wabc radio weight loss sponsor http://us.startv.com/node/7/downloads eygptian diet http://us.startv.com/gallery/neil-deepika-chhote-ustaad-3 deadweight welfare loss http://us.startv.com/wallpapers/humari-devrani-wallpaper free weightloss journal http://us.startv.com/news/vivian-speaks-about-abhay directories for submission http://us.startv.com/wallpapers/masterchef cardio burns fat http://us.startv.com/recaps/masterchef weightloss camps for kids http://us.startv.com/wallpapers/masterchef-india-0 japanese macrobiotic diet http://us.startv.com/gallery/lkp-2 submit your masturbation technique http://us.startv.com/synopsis/ajab diete proteinee

comment6 http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewMonth&tx_teksaccext_pi4[month]=092010 healthy low calorie diets http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=2852&tx_teksaccext_pi3[address]= robotics equipment submit your site http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewMonth&tx_teksaccext_pi4[month]=021801&tx_teksaccext_pi4[expandEvents]=true&tx_teksaccext_pi4[category]=12 mike mahler diet http://www.springfieldchamber.com/index.php?id=124&no_cache=1 with the adkins diet http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=emailForm&tx_teksaccext_pi3[uid]=294 submission erotic story http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewMonth&tx_teksaccext_pi4[month]=102010&tx_teksaccext_pi4[expandEvents]=true diet soda phosphorus peyronies syndrome http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewMonth&tx_teksaccext_pi4[month]=021792 acid alkalin diet http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=1517&tx_teksaccext_pi3[address]= blood type diet food lists http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=2920&tx_teksaccext_pi3[address]= does the diet solution work http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=6168&tx_teksaccext_pi3[address]= $153 weight loss pill http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=1110&tx_teksaccext_pi3[address]= diet food raw start http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewdate&tx_teksaccext_pi4[date]=1287464400 com diet healthy loss newnitetrim program weight http://www.springfieldchamber.com/no_cache/news_updates/calendar_and_special_events/?tx_teksaccext_pi4[mode]=viewMonth&tx_teksaccext_pi4[month]=021801&tx_teksaccext_pi4[expandEvents]=true&tx_teksaccext_pi4[category]=5 individual states bmi http://www.springfieldchamber.com/no_cache/business_directory/business_a_z/?tx_teksaccext_pi3[mode]=redirWWW&tx_teksaccext_pi3[uid]=2174&tx_teksaccext_pi3[address]= quick weight loss centers cost http://www.springfieldchamber.com/no_cache/business_directory/?tx_teksaccext_pi1[mode]=redirWWW&tx_teksaccext_pi1[uid]=5374&tx_teksaccext_pi1[address]= heart medical hospital diet

Test the exports for correctness if imports fail

The imports will fail if the exports are not correct. Use the following to setup a script to check the exports for correctness on the new server.

Create export check script

Open the export check script for editing as follows: 

nano /root/scripts/collection-check

Copy and paste the following to open editor: 

#!/bin/bash

# Setup the configs.
. config

# Clean out the old logs.
rm /root/test.log

# Go thru the collections
for i in `cat collections` ; do
    echo $i > /tmp/map
    ECID=`cat /tmp/map | awk -F',' '{ print $1 }'` 
    ICID=`cat /tmp/map | awk -F',' '{ print $2 }'`
    echo "Checking items in collection:$ECID"
    LIST=`ls $IMPORT/$ECID`
    
    for i in $LIST ; do
	ITM=$i
	cd  $IMPORT/$ECID/$ITM
	if [ ! -e dublin_core.xml ] ; then
	echo "Item: $ITM, No dublin core." >> /root/test.log
	fi
	if [ ! -e handle ] ; then
	echo "Item: $ITM, No handle." >> /root/test.log
	fi
	if [ ! -e contents ] ; then
	echo "Item: $ITM, No contents file." >> /root/test.log
	fi
	if [ ! -e license.txt ] ; then
	echo "Item: $ITM, No license." >> /root/test.log
	fi
	ls $IMPORT/$ECID/$ITM | grep pdf > /tmp/pdf.log
	if [ $? -ne 0 ] ; then
	echo "Item: $ITM, No PDF file." >> /root/test.log
	fi
    done
    cat /root/test.log | mail -s "Exported collections test log" $PERSON
    echo "Check complete and report emailed to dspace admin user"
done

Save the file and exit the editor.

Now make the script executeable as follows: 

chmod 0775 /root/scripts/collection-check

Do the exports check

Type the following to do the check: 

/root/scripts/collection-check

Now check your email, you should have one with details of the test log.

If there are errors, fix them on the old server. Delete the exports on the old server and do a new export after everything is fixed. Bascially start the export and import over again. At least you now have the scripts to do it over again.

Help Info

/home/dspace/bin/export

It takes the following parameters: 

-d,--dest     destination collection directory
-h,--help     help
-i,--id       collection/item ID
-n,--number   sequence number to begin exporting items with
-m,--migrate  strips out any per-repository-specifics
-t,--type     type: COLLECTION or ITEM

/home/dspace/bin/import

It takes the following parameters: 

-s,--source        source collection directory
-c,--collection    destination collection handle
-e,--eperson       email of authorised person
-m,--mapfile       collection/item map file name 
-a,--add           add items
-d,--delete        delete items listed in mapfile
-r,--replace       replace items listed in mapfile 
-w,--workflow      send submission through collection's workflow
-p,--template      apply template
-t,--test          test run - do not actually import items
-R,--resume        resume a failed import (add only)
-h,--help          help

References

Command Line Help

Go to: http://www.ubuntu.sun.ac.za/wiki/index.php/SelfHelp for more help about the command line programs used in this procedure. 

Back to IR Help
Retrieved from "http://wiki.lib.sun.ac.za/index.php?title=SUNScholar/Export_and_Import_Artifacts&oldid=6804"