SUNScholar/Optimisations

Requirements

This wiki help page assumes that you have used the three system setup procedures to install an Ubuntu server with DSpace software.

Introduction

This wiki page details the major optimisations of the system performed at Stellenbosch University in order to create a truly production version of DSpace.

Reduce system log sizes

Click on the heading above.

Email error logs

Click on the heading above

Java Webapp Performance

Perform Java profiling to find memory and CPU intensive code.

Information

• http://en.wikipedia.org/wiki/Java_performance
• http://en.wikipedia.org/wiki/VisualVM
• http://en.wikipedia.org/wiki/Java_Development_Kit
• http://en.wikipedia.org/wiki/OpenJDK

Tools

• http://java-source.net/open-source/profilers
• http://www.infoq.com/articles/java-profiling-with-open-source
• http://theholyjava.wordpress.com/2012/02/25/profiling-tomcat-webapp-with-visualvm-and-netbeans-pitfalls
• http://java.dzone.com/articles/pitfalls-experienced-when
• http://docs.oracle.com/javase/6/docs/technotes/guides/visualvm/profiler.html
• http://www.yourkit.com

Tomcat

See: http://www.turnkeylinux.org/tomcat

• Remove "mod_jk", use "authbind" exclusively with no need of the Tomcat AJP connector in order to reduce the CPU and memory load

http://wiki.lib.sun.ac.za/index.php/SUNScholar/Prepare_Ubuntu/S05

• Remove "development mode" of Tomcat by adding the following to the server context in order to reduce DNS lookups.

enableLookups="false"

XMLUI

• Use XMLUI exclusively to reduce the memory load.

http://wiki.lib.sun.ac.za/index.php/SUNScholar/Install_Dspace/S08

Database

• Fix "browse index" configuration to reduce the PostgreSQL database server query load.

http://wiki.lib.sun.ac.za/index.php/SUNScholar/Indexes#Browse_Indexes

Vacumn the database regularly

http://wiki.lib.sun.ac.za/index.php/SUNScholar/Daily_Admin#.22postgres.22_crontab

Bitstream checker

Modified bitstream checker settings as follows to reduce database size.

#### Checksum Checker Settings ####
# Default dispatcher in case none specified
plugin.single.org.dspace.checker.BitstreamDispatcher=org.dspace.checker.SimpleDispatcher

# check history retention
checker.retention.default=1y
checker.retention.CHECKSUM_MATCH=2w

Monit monitor service

In case the Tomcat service halts or hangs due to whatever... , I installed monit to restart the service and then alert me. See an example of my config below.

dspace@ir1:/etc/monit$ sudo cat /etc/monit/monitrc
set daemon  60
set logfile syslog facility log_daemon
set mailserver localhost
set mail-format { from: XXXX@XX.XXX.XX.XX }
set alert XXXX@localhost
set httpd port 2812
     allow %user%:%password%

check process sshd with pidfile /var/run/sshd.pid
   start program  "/etc/init.d/ssh start"
   stop program  "/etc/init.d/ssh stop"
   if failed port 22 protocol ssh then restart
   
check host sunscholar with address scholar.sun.ac.za
   start program = "/etc/init.d/tomcat6 restart"
   stop program  = "/etc/init.d/tomcat6 stop"
   if failed port 80 proto http then restart
   alert XXXX@XXX.XX.XX
   alert XXXX@XXX.XX.XX

All confidential information has been replaced with % signs or captial X's.

References

• https://wiki.duraspace.org/display/DSDOC18/Performance+Tuning+DSpace
• http://www.turnkeylinux.org/tomcat

---

All our tweaks and optimisations seem to be working. The load dropped when we started using "authbind" for Tomcat thereby eliminating the need for the Apache "mod_jk" module, which was creating extra processing overhead.	Looks like we have enough disk space in the /home partition for the next 3yrs at least, at our current rate of submissions. The /var partition which holds the database was reduced in size by tweaking the bitstream checker properties and then running a full database vacuum.
We have more than enough compute muscle.	Our memory usage stabilised when we stopped using the JSPUI. However after the upgrade to DSpace 1.8.2 and enabling discovery we are back to a memory intensive system.

Back to After Installation Tasks