SUNScholar/Daily Admin

From Libopedia
Jump to navigation Jump to search
Back to After Installation Tasks

Step 1. Login

http://wiki.lib.sun.ac.za/index.php/SUNScholar/Prepare_Ubuntu/S01

Click on the link above to find out how to login to the server and then return here.

Step 2. Create "dspace" user crontab

The "dspace" user has to perform tasks automatically at regular intervals, such as sending out subscription emails.

On a Linux/Unix system this is easy to accomplish using the "cron" functionality.

Sample crontab for DSpace => 3.2

Refer to original at: https://github.com/DSpace/demo.dspace.org/blob/master/scripts/linux/crontab

Edit the crontab, by typing the following in a terminal:

su - dspace
crontab -e

If asked to select an editor, choose nano

Instructions
  1. Delete all the contents of the file using CTL+k repeatedly
  2. Copy and paste the following into the open file.
  3. Save the file by typing CTL+o
  4. Exit the file by typing CTL+x
####################################
# Initialize Environment Variables #
####################################
# Deliver cron email to system administrator
MAILTO="%myadmin@repo.ac.za%"

#Ensure that HOME is set properly for our service
HOME=/home/dspace

#Add Java to PATH (for all DSpace cron jobs)
#Also add all major 'bin' directories
PATH=/usr/bin:/bin:/usr/local/bin

#Specify default Java options (for all DSpace cron jobs)
JAVA_OPTS=-Xmx512M -Xms512M -Dfile.encoding=UTF-8

##########
# Hourly #
##########
# Regenerate System Sitemaps every 8 hours
# at 3 minutes past the hour
3  0,8,16 * * * $HOME/bin/dspace generate-sitemaps > /dev/null

# Update item counts every hour
@hourly	   $HOME/bin/dspace itemcounter > /dev/null

###########
# Nightly #
###########
# 12:30AM
# Run DSpace statistical analysis tools (12months takes approx 40secs)
30 0 * * * $HOME/bin/dspace stat-general  > /dev/null
35 0 * * * $HOME/bin/dspace stat-monthly  > /dev/null

# 1:00AM
# Generate DSpace statistical analysis reports
00 1 * * * $HOME/bin/dspace stat-report-general  > /dev/null
05 1 * * * $HOME/bin/dspace stat-report-monthly  > /dev/null

# 2:00AM
# Run the DSpace media filter scripts and save all output to a log file
# (Creates image thumbnails, indexes full text, and 
# performs any automated format conversions)
0 2 * * * $HOME/bin/dspace filter-media > $HOME/log/media-filter.log 2>&1

# 3:00AM
# Update simple, advanced search and browse indexes
0 3 * * * $HOME/bin/dspace index-update > /dev/null

# 5:00AM
# Check for items to release from embargo in DSpace.  
0 5 * * * $HOME/bin/dspace embargo-lifter > $HOME/log/embargo-release.log 2>&1

# 6:00AM
# Run XOAI incremental import (and optimization)
0 6 * * * $HOME/bin/dspace oai import -o > /dev/null

# 6:30AM
# Optimise Discovery Solr Index
30 6 * * * $HOME/bin/dspace update-discovery-index -o > /dev/null
# Cleanup Web Spiders from DSpace Statistics Solr Index
# -i deletes all spiders matched by IP address, DNS name or Agent name
30 6 * * * $HOME/bin/dspace stats-util -i > /dev/null

# 7:00AM
# Optimize DSpace Statistics Solr Index
0 7 * * * $HOME/bin/dspace stats-util -o > /dev/null

# 8:00AM
# Send out DSpace subscription emails
# (This alerts users of newly deposited items of interest)
0 8 * * * $HOME/bin/dspace sub-daily > /dev/null

##########
# Weekly #
##########

###########
# Monthly #
###########
# 12:01AM
# Remove all DSpace log files which are more than 30 days old
# on the first of every month
01 0 1 * * find $HOME/log/*.log.* -mtime +30 -exec rm {} \;

# 1:00AM
# Completely remove any deleted bitstreams in DSpace
# on the first of every month
0 1 1 * * $HOME/bin/dspace cleanup > /dev/null

# 2:00AM
# Delete any ~/dspace/config/*/*.old files more than 30 days old (created by "ant update")
0 2 1 * * find $HOME/config -name "*-*-*.old" -mtime +30 -exec rm {} \;
# Delete any ~/dspace/*.bak-*-*/ directories more than 30 days old (created by "ant update")
0 2 1 * * find $HOME/*.bak-*-* -maxdepth 0 -type d -mtime +30 -exec rm -rf {} \;

NANO Editor Help
CTL+O = Save the file and then press Enter
CTL+X = Exit "nano"
CTL+K = Delete line
CTL+U = Undelete line
CTL+W = Search for %%string%%
CTL+\ = Search for %%string%% and replace with $$string$$
CTL+C = Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)


Sample crontab for DSpace <= 1.8.2 (superseded, see above)

Edit the crontab, by typing the following in a terminal:

su - dspace
crontab -e

If asked to select an editor, choose nano

Remove all the contents by pressing CTL+k repeatedly, then add the following lines using the copy and paste functions of firefox and nano:

Replace %myadmin@repo.ac.za% with the email address of the person to receive system admin email.

# m h  dom mon dow   command
MAILTO="%myadmin@repo.ac.za%"

### Setup embargoes before enabling this ###
### After embargoes are setup, remove the hash to enable embargo checking ###
### See wiki customisation page for setup details ###
### http://wiki.lib.sun.ac.za/index.php/SUNScholar/Embargo_Systems ###
## Check for lifted embargos ##
#05 0 * * *      /home/dspace/bin/dspace embargo-lifter -a -v
##The above option is made available with DSpace 3.2 only
#10 0 * * *      /home/dspace/bin/dspace embargo-lifter -l -v
#15 0 * * *      /home/dspace/bin/dspace embargo-lifter -c -v

### See wiki customisation page for setup details ###
### http://wiki.lib.sun.ac.za/index.php/SUNScholar/Browse_Indexes ###
## Update item search and browse indexes ##
0 3 * * *       /home/dspace/bin/dspace index-update

### See wiki customisation page for setup details ###
### http://wiki.lib.sun.ac.za/index.php/SUNScholar/SOLR_Statistics ##
## SOLR stats update ##
@daily          /home/dspace/bin/dspace stats-util -b -r   
@weekly         /home/dspace/bin/dspace stats-util -o
@monthly        /home/dspace/bin/dspace stats-util -u -m -i -f

### See wiki customisation page for setup details ###
### http://wiki.lib.sun.ac.za/index.php/SUNScholar/SOLR_Discovery ###
## SOLR discovery update ##
@daily          /home/dspace/bin/dspace update-discovery-index
@weekly         /home/dspace/bin/dspace update-discovery-index -o
@monthly        /home/dspace/bin/dspace update-discovery-index -c

## Generate sitemaps for Google indexing ##
0 6 * * *       /home/dspace/bin/dspace generate-sitemaps

## Item counts ##
@hourly		/home/dspace/bin/dspace itemcounter

## Send out subscription e-mails ##
0 1 * * *       /home/dspace/bin/dspace sub-daily
 
## Run the media filter ##
## This is setup to run weekly to reduce server load ##
## It can be setup to run nightly if you want ##
@weekly	        /home/dspace/bin/dspace filter-media

## Bitstream checks ##
@weekly         /home/dspace/bin/dspace checker-emailer -a
@monthly        /home/dspace/bin/dspace checker -p -l
@monthly  	/home/dspace/bin/dspace cleanup -v

### The following are not needed if using the new SOLR stats above ###
## Run stat analyses ##
#10 1 * * *      /home/dspace/bin/dspace stat-general
#20 1 * * *      /home/dspace/bin/dspace stat-monthly

## Run stat analyses reports ##
#10 2 * * *      /home/dspace/bin/dspace stat-report-general
#20 2 * * *      /home/dspace/bin/dspace stat-report-monthly

NANO Editor Help
CTL+O = Save the file and then press Enter
CTL+X = Exit "nano"
CTL+K = Delete line
CTL+U = Undelete line
CTL+W = Search for %%string%%
CTL+\ = Search for %%string%% and replace with $$string$$
CTL+C = Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)


Step 3. Create "postgres" user crontab

PostgreSQL also benefits from regular 'vacuuming', which optimizes the indexes and clears out any deleted data.

Type as follows in a terminal:

sudo -i

Enter the "dspace" user password. Then type:

su - postgres
crontab -e

Copy and paste the following:

20 4 * * * vacuumdb --analyze dspace > /dev/null 2>&1

NANO Editor Help
CTL+O = Save the file and then press Enter
CTL+X = Exit "nano"
CTL+K = Delete line
CTL+U = Undelete line
CTL+W = Search for %%string%%
CTL+\ = Search for %%string%% and replace with $$string$$
CTL+C = Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)


References