SUNScholar/Optimisations/Logs

From Libopedia
Jump to: navigation, search
Back to Optimisations

Troubleshooting Guide

Also see: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Optimisations/Troubleshooting

Default Log Levels

With the release of DSpace 4.2, it is now possible to define default log levels.

Modif the the /home/dspace/source/build.properties file. See example below.

#####################
# LOGLEVEL SETTINGS #
#####################
loglevel.other = INFO
# loglevel.other: Log level for other third-party tools/APIs used by DSpace
# Possible values (from most to least info): DEBUG, INFO, WARN, ERROR, FATAL
loglevel.dspace = INFO
# loglevel.dspace: Log level for all DSpace-specific code (org.dspace.*)
# Possible values (from most to least info): DEBUG, INFO, WARN, ERROR, FATAL

Tomcat Java web applications

Changed all instances of "INFO" to "ERROR" in the following config file to reduce disk I/O and CPU load.

nano /home/dspace/source/dspace/config/log4j.properties

See example below:

###########################################################################
#
# log4j.properties
#
# Version: $Revision$
#
# Date: $Date$
#
###########################################################################

# This is the log4j configuration file for DSpace. Log4j can be configured to
# write files that rotate every day or every month.
#
# Text surrounded by ${} symbols is replaced with the corresponding
# property from dspace.cfg by Ant when DSpace is deployed.  For example:
#
# dspace.url
#
# would be replaced with the dspace.url property in dspace.cfg.
#
# Additional properties expanded by Maven during the DSpace assembly
# process:
# 
# loglevel.dspace (currently set to: ${loglevel.dspace})
#   Log level for all DSpace-specific code (org.dspace.*)
#   Possible values (from most to least info): DEBUG, INFO, WARN, ERROR, FATAL
#
# loglevel.other (currently set to: ${loglevel.other})
#   Log level for other third-party tools/APIs used by DSpace
#   Possible values (from most to least info): DEBUG, INFO, WARN, ERROR, FATAL

###########################################################################
# A1 is the name of the appender for most DSpace activity.
###########################################################################
# The root category is the default setting for all non-DSpace code.
# Change this from INFO to DEBUG to see extra logging created by non-DSpace
# code.
log4j.rootCategory=${loglevel.other}, A1
# This line sets the logging level for DSpace code. Set this to DEBUG to see
# extra detailed logging for DSpace code.
log4j.logger.org.dspace=${loglevel.dspace}, A1
# Do not change this line
log4j.additivity.org.dspace=false
# The name of the file appender
log4j.appender.A1=org.dspace.app.util.DailyFileAppender
# The filename of the log file created. A date stamp is appended to this
log4j.appender.A1.File=${log.dir}/dspace.log
# Set this to yyyy-MM-DD for daily log files, or yyyy-MM for monthly files
log4j.appender.A1.DatePattern=yyyy-MM-dd
# The number of log files to keep, or 0 to keep them all
log4j.appender.A1.MaxLogs=0
# A1 uses PatternLayout.
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%d %-5p %c @ %m%n


###########################################################################
# A2 is the name of the appender for the Checker
###########################################################################
# This line sets the logging level for the checksum checker log file.
# Set this to DEBUG to see extra detailed logging.
log4j.logger.org.dspace.checker=INFO, A2
# Do not change this line
log4j.additivity.org.dspace.checker=false
# The name of the file appender
log4j.appender.A2=org.dspace.app.util.DailyFileAppender
# The filename of the log file created. A date stamp is appended to this
log4j.appender.A2.File=${log.dir}/checker.log
# Set this to yyyy-MM-DD for daily log files, or yyyy-MM for monthly files
log4j.appender.A2.DatePattern=yyyy-MM-dd
# The number of log files to keep, or 0 to keep them all
log4j.appender.A2.MaxLogs=0
# A2 uses PatternLayout.
log4j.appender.A2.layout=org.apache.log4j.PatternLayout
log4j.appender.A2.layout.ConversionPattern=%m%n


###########################################################################
# A3 is the name of the appender for Cocoon (XMLUI only)
###########################################################################
# These lines sets the logging level for the cocoon log file.
# Set these to DEBUG to see extra detailed logging.
log4j.logger.org.apache.cocoon=INFO, A3
log4j.logger.cocoon=INFO, A3
log4j.logger.org.springframework=INFO, A3
# Do not change these lines
log4j.additivity.org.apache.cocoon=false
log4j.additivity.cocoon=false
log4j.additivity.org.springframework=false
# The name of the file appender
log4j.appender.A3=org.dspace.app.util.DailyFileAppender
# The filename of the log file created. A date stamp is appended to this
log4j.appender.A3.File=${log.dir}/cocoon.log
# Set this to yyyy-MM-DD for daily log files, or yyyy-MM for monthly files
log4j.appender.A3.DatePattern=yyyy-MM-dd
# The number of log files to keep, or 0 to keep them all
log4j.appender.A3.MaxLogs=14
# A2 uses PatternLayout.
log4j.appender.A3.layout=org.apache.log4j.PatternLayout
log4j.appender.A3.layout.ConversionPattern=%d %-5p %c %x - %m%n


###########################################################################
# A4 is the name of the appender for Solr
###########################################################################
log4j.logger.org.apache.solr=ERROR, A4
log4j.additivity.org.apache.solr=false
log4j.appender.A4=org.dspace.app.util.DailyFileAppender
log4j.appender.A4.File=${log.dir}/solr.log
log4j.appender.A4.DatePattern=yyyy-MM-dd
log4j.appender.A4.MaxLogs=14
log4j.appender.A4.layout=org.apache.log4j.PatternLayout
log4j.appender.A4.layout.ConversionPattern=%d %-5p %c %x - %m%n


###########################################################################
# Other settings
###########################################################################

# Block passwords from being exposed in Axis logs.
# (DEBUG exposes passwords in Basic Auth)
log4j.logger.org.apache.axis.handlers.http.HTTPAuthHandler=INFO

# Block services logging except on exceptions
log4j.logger.org.dspace.kernel=ERROR
log4j.logger.org.dspace.services=ERROR
log4j.logger.org.dspace.servicemanager=ERROR
log4j.logger.org.dspace.providers=ERROR
log4j.logger.org.dspace.utils=ERROR

Then rebuild DSpace.

SOLR

Log level

Please note: The procedure below is not stable, so I installed PSI Probe using the Tomcat manager web interface and now I set the SOLR log level via the web gui after each re-build. See: https://code.google.com/p/psi-probe

See: http://lucidworks.lucidimagination.com/display/solr/Configuring+Logging and https://jira.duraspace.org/browse/DS-1460

Create the following file:

nano /home/dspace/source/dspace/modules/solr/src/main/webapp/WEB-INF/classes/logging.properties

Add the following to the file:

 org.apache.commons.digester.Digester.level = WARNING
 org.apache.solr.level = SEVERE

NANO Editor Help
CTL+O = Save the file and then press Enter
CTL+X = Exit "nano"
CTL+K = Delete line
CTL+U = Undelete line
CTL+W = Search for $string$
CTL+\ = Search for and replace $string$
ALT+C = Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)


Save the file and rebuild.

Also see: http://skybert.wordpress.com/2009/07/22/how-to-get-solr-to-log-to-a-log-file

See news from @atmire below.

Limiting log output for SOLR

Adding SOLR logging propertiesWe extensively covered the growth of log files and the different log levels in last month's article. In addition to the dspace logfiles contained in /dspace/log, some DSpace components write directly into the logfiles of your application container, Tomcat. One culprit in particular is SOLR. SOLR is the key component responsible for statistics, Discovery and most recently, OAI-PMH. The default logging level for SOLR is INFO, meaning that it will write to the Tomcat catalina.out logfile for each request it receives. To override this behavior add a file named logging.properties to the solr module found in $dspace.src$/dspace/modules/solr/src/main/resources.

Add the following instruction in this empty file:

org.apache.solr.level = WARNING

Log file permissions

I setup a root cron crontab as follows.

sudo -i
crontab -e

Add the following line.

@midnight   chmod 0777 -R /home/dspace/log

Save the crontab edit.


NANO Editor Help
CTL+O = Save the file and then press Enter
CTL+X = Exit "nano"
CTL+K = Delete line
CTL+U = Undelete line
CTL+W = Search for $string$
CTL+\ = Search for and replace $string$
ALT+C = Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)


See: http://wiki.lib.sun.ac.za/index.php/SUNScholar/Install_DSpace/S06#Step_6.3:_Setup_Tomcat_server_permissions for reference about file permissions.