Difference between revisions of "SUNScholar/Daily Admin"

From Libopedia
Jump to navigation Jump to search
Line 4: Line 4:
 
</center>
 
</center>
  
−
==Step 1. Login==
+
==[[SUNScholar/Daily Admin/3.2|For DSpace 3.2]]==
−
http://wiki.lib.sun.ac.za/index.php/SUNScholar/Prepare_Ubuntu/S01
+
Click on the heading above.
−
Click on the link above to find out how to login to the server and then return here.
+
   
−
 
 
−
==Step 2. Create "dspace" user crontab==
 
−
The "dspace" user has to perform tasks automatically at regular intervals, such as sending out subscription emails.
 
−
 
 
−
On a Linux/Unix system this is easy to accomplish using the "cron" functionality.
 
−
 
 
−
===Sample crontab for DSpace => 3.2===
 
−
Refer to original at: '''https://github.com/DSpace/demo.dspace.org/blob/master/scripts/linux/crontab'''
 
−
 
 
−
Edit the crontab, by typing the following in a terminal:
 
−
 
 
−
su - dspace
 
−
 
 
−
crontab -e
 
−
If asked to select an editor, choose '''nano'''
 
−
 
 
−
;Instructions
 
−
# Delete all the contents of the file using '''CTL+k''' repeatedly
 
−
# Copy and paste the following into the open file.
 
−
# Save the file by typing '''CTL+o'''
 
−
# Exit the file by typing '''CTL+x'''
 
−
<pre>
 
−
####################################
 
−
# Initialize Environment Variables #
 
−
####################################
 
−
# Deliver cron email to system administrator
 
−
MAILTO="root"
 
−
 
 
−
#Ensure that HOME is set properly for our service
 
−
HOME=/home/dspace
 
−
 
 
−
#Add Java to PATH (for all DSpace cron jobs)
 
−
#Also add all major 'bin' directories
 
−
PATH=/usr/bin:/bin:/usr/local/bin
 
−
 
 
−
#Specify default Java options (for all DSpace cron jobs)
 
−
JAVA_OPTS=-Xmx512M -Xms512M -Dfile.encoding=UTF-8
 
−
 
 
−
##########
 
−
# Hourly #
 
−
##########
 
−
# Regenerate System Sitemaps every 8 hours
 
−
# at 3 minutes past the hour
 
−
3  0,8,16 * * * $HOME/bin/dspace generate-sitemaps > /dev/null
 
−
 
 
−
# Update item counts every hour
 
−
@hourly     $HOME/bin/dspace itemcounter > /dev/null
 
−
 
 
−
###########
 
−
# Nightly #
 
−
###########
 
−
# 12:30AM
 
−
# Run DSpace statistical analysis tools (12months takes approx 40secs)
 
−
30 0 * * * $HOME/bin/dspace stat-general > /dev/null
 
−
35 0 * * * $HOME/bin/dspace stat-monthly  > /dev/null
 
−
 
 
−
# 1:00AM
 
−
# Generate DSpace statistical analysis reports
 
−
00 1 * * * $HOME/bin/dspace stat-report-general  > /dev/null
 
−
05 1 * * * $HOME/bin/dspace stat-report-monthly  > /dev/null
 
−
 
 
−
# 2:00AM
 
−
# Run the DSpace media filter scripts and save all output to a log file
 
−
# (Creates image thumbnails, indexes full text, and
 
−
# performs any automated format conversions)
 
−
0 2 * * * $HOME/bin/dspace filter-media > $HOME/log/media-filter.log 2>&1
 
−
 
 
−
# 3:00AM
 
−
# Update simple, advanced search and browse indexes
 
−
0 3 * * *  $HOME/bin/dspace index-update > /dev/null
 
−
 
 
−
# 5:00AM
 
−
# Check for items to release from embargo in DSpace. 
 
−
0 5 * * * $HOME/bin/dspace embargo-lifter > $HOME/log/embargo-release.log 2>&1
 
−
 
 
−
# 6:00AM
 
−
# Run XOAI incremental import (and optimization)
 
−
0 6 * * * $HOME/bin/dspace oai import -o > /dev/null
 
−
 
 
−
# 6:30AM
 
−
# Cleanup Web Spiders from DSpace Statistics Solr Index
 
−
# -i deletes all spiders matched by IP address, DNS name or Agent name
 
−
30 6 * * * $HOME/bin/dspace stats-util -i > /dev/null
 
−
 
 
−
# 7:00AM
 
−
# Optimize Discovery Solr Index
 
−
0 7 * * * $HOME/bin/dspace update-discovery-index -o > /dev/null
 
−
 
 
−
# 7:30AM
 
−
# Optimize DSpace Statistics Solr Index
 
−
30 7 * * * $HOME/bin/dspace stats-util -o > /dev/null
 
−
 
 
−
# 8:00AM
 
−
# Send out DSpace subscription emails
 
−
# (This alerts users of newly deposited items of interest)
 
−
0 8 * * * $HOME/bin/dspace sub-daily > /dev/null
 
−
 
 
−
##########
 
−
# Weekly #
 
−
##########
 
−
 
 
−
###########
 
−
# Monthly #
 
−
###########
 
−
# 12:01AM
 
−
# Remove all DSpace log files which are more than 30 days old
 
−
# on the first of every month
 
−
01 0 1 * * find $HOME/log/*.log.* -mtime +30 -exec rm {} \;
 
−
 
 
−
# 1:00AM
 
−
# Completely remove any deleted bitstreams in DSpace
 
−
# on the first of every month
 
−
0 1 1 * * $HOME/bin/dspace cleanup > /dev/null
 
−
 
 
−
# 2:00AM
 
−
# Delete any ~/config/*/*.old files more than 30 days old (created by "ant update")
 
−
0 2 1 * * find $HOME/config -name "*-*-*.old" -mtime +30 -exec rm {} \;
 
−
# Delete any ~/*.bak-*-*/ directories more than 30 days old (created by "ant update")
 
−
0 2 1 * * find $HOME/*.bak-*-* -maxdepth 0 -type d -mtime +30 -exec rm -rf {} \;
 
−
</pre>
 
−
 
 
−
{{NANO}}
 
  
 
===Sample crontab for DSpace <= 1.8.2 (superseded, see above)===
 
===Sample crontab for DSpace <= 1.8.2 (superseded, see above)===

Revision as of 10:38, 6 February 2014

Back to After Installation Tasks

For DSpace 3.2

Click on the heading above.


Sample crontab for DSpace <= 1.8.2 (superseded, see above)

Edit the crontab, by typing the following in a terminal: 

su - dspace

crontab -e

If asked to select an editor, choose nano

Remove all the contents by pressing CTL+k repeatedly, then add the following lines using the copy and paste functions of firefox and nano:

Replace %myadmin@repo.ac.za% with the email address of the person to receive system admin email. 

# m h  dom mon dow   command
MAILTO="%myadmin@repo.ac.za%"

### Setup embargoes before enabling this ###
### After embargoes are setup, remove the hash to enable embargo checking ###
### See wiki customisation page for setup details ###
### http://wiki.lib.sun.ac.za/index.php/SUNScholar/Embargo_Systems ###
## Check for lifted embargos ##
#05 0 * * *      /home/dspace/bin/dspace embargo-lifter -a -v
##The above option is made available with DSpace 3.2 only
#10 0 * * *      /home/dspace/bin/dspace embargo-lifter -l -v
#15 0 * * *      /home/dspace/bin/dspace embargo-lifter -c -v

### See wiki customisation page for setup details ###
### http://wiki.lib.sun.ac.za/index.php/SUNScholar/Browse_Indexes ###
## Update item search and browse indexes ##
0 3 * * *       /home/dspace/bin/dspace index-update

### See wiki customisation page for setup details ###
### http://wiki.lib.sun.ac.za/index.php/SUNScholar/SOLR_Statistics ##
## SOLR stats update ##
@daily          /home/dspace/bin/dspace stats-util -b -r   
@weekly         /home/dspace/bin/dspace stats-util -o
@monthly        /home/dspace/bin/dspace stats-util -u -m -i -f

### See wiki customisation page for setup details ###
### http://wiki.lib.sun.ac.za/index.php/SUNScholar/SOLR_Discovery ###
## SOLR discovery update ##
@daily          /home/dspace/bin/dspace update-discovery-index
@weekly         /home/dspace/bin/dspace update-discovery-index -o
@monthly        /home/dspace/bin/dspace update-discovery-index -c

## Generate sitemaps for Google indexing ##
0 6 * * *       /home/dspace/bin/dspace generate-sitemaps

## Item counts ##
@hourly		/home/dspace/bin/dspace itemcounter

## Send out subscription e-mails ##
0 1 * * *       /home/dspace/bin/dspace sub-daily
 
## Run the media filter ##
## This is setup to run weekly to reduce server load ##
## It can be setup to run nightly if you want ##
@weekly	        /home/dspace/bin/dspace filter-media

## Bitstream checks ##
@weekly         /home/dspace/bin/dspace checker-emailer -a
@monthly        /home/dspace/bin/dspace checker -p -l
@monthly  	/home/dspace/bin/dspace cleanup -v

### The following are not needed if using the new SOLR stats above ###
## Run stat analyses ##
#10 1 * * *      /home/dspace/bin/dspace stat-general
#20 1 * * *      /home/dspace/bin/dspace stat-monthly

## Run stat analyses reports ##
#10 2 * * *      /home/dspace/bin/dspace stat-report-general
#20 2 * * *      /home/dspace/bin/dspace stat-report-monthly
NANO Editor Help
CTL+O = Save the file and then press Enter
CTL+X = Exit "nano"
CTL+K = Delete line
CTL+U = Undelete line
CTL+W = Search for %%string%%
CTL+\ = Search for %%string%% and replace with $$string$$
CTL+C = Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)

Step 3. Create "postgres" user crontab

PostgreSQL also benefits from regular 'vacuuming', which optimizes the indexes and clears out any deleted data.

Type as follows in a terminal: 

sudo -i

Enter the "dspace" user password. Then type: 

su - postgres

crontab -e

Copy and paste the following: 

20 4 * * * vacuumdb --analyze dspace > /dev/null 2>&1
NANO Editor Help
CTL+O = Save the file and then press Enter
CTL+X = Exit "nano"
CTL+K = Delete line
CTL+U = Undelete line
CTL+W = Search for %%string%%
CTL+\ = Search for %%string%% and replace with $$string$$
CTL+C = Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)

References

Retrieved from "http://wiki.lib.sun.ac.za/index.php?title=SUNScholar/Daily_Admin&oldid=25087"