Difference between revisions of "SUNScholar/Disaster Recovery/Backups/Local"

From Libopedia
Jump to navigation Jump to search
 
(38 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
==Step 1. Setup local daily backup==
+
<center>
 +
'''[[SUNScholar/Disaster Recovery/Backups/Client Setup|Back to Client Setup]]'''
 +
</center>
 +
 
 +
<font color="red">'''First [[SUNScholar/Disaster_Recovery/Preparation|click here]] to create a PostgreSQL credentials file.'''</font>
 +
 
 
===Create local backup script===
 
===Create local backup script===
 
Here is a sample script to make local backups which are then sent to the remote backup server.
 
Here is a sample script to make local backups which are then sent to the remote backup server.
Line 6: Line 11:
 
  sudo nano /usr/local/bin/backup.sh
 
  sudo nano /usr/local/bin/backup.sh
 
Now copy and paste the following into the open editor and modify the backup variables to suit your location and server.
 
Now copy and paste the following into the open editor and modify the backup variables to suit your location and server.
 +
 +
#{{HOSTNAME}}
 +
#Replace '''%email-address%''' with the email address of the person responsible for server backups.
 +
 
<pre>
 
<pre>
 
#!/bin/bash
 
#!/bin/bash
  
−
## Setup the backup variables ##
+
# Setup shell to use for backups
−
LOCAL_SERVER="etd.sun.ac.za"
+
SHELL=/bin/bash
−
LOCAL_FOLDER="/var/backup" 
+
 
−
BACKUP_LOGFILE="/var/log/backup.log"
+
# Setup name of local server to be backed up
−
# Day Of the Week
+
SERVER="%hostname%"
 +
 
 +
# Setup event stamps
 
DOW=`date +%a`
 
DOW=`date +%a`
 
TIME=`date`
 
TIME=`date`
  
 +
# Setup paths
 +
FOLDER="/home/backup"
 +
FILE="/var/log/backup-$DOW.log"
 +
 +
# Do the backups
 
{
 
{
−
## Timestamp the beginning of the backup ##
+
echo "Backup started: $TIME"
−
echo "Backup for $LOCAL_SERVER started: $TIME"
 
  
−
## Check that we have a backup folder ##
+
# Make the backup folder if it does not exist
−
if [ ! -d $LOCAL_FOLDER ]; then
+
if test ! -d /home/backup
−
   mkdir -p $LOCAL_FOLDER
+
then
 +
   mkdir -p /home/backup
 
   echo "New backup folder created"
 
   echo "New backup folder created"
 
   else
 
   else
−
   echo "Backup started: $TIME"
+
   echo ""
 
fi
 
fi
  
−
## Make sure we're in / since backups are relative to that ##
+
# Make sure we're in / since backups are relative to that
 
cd /
 
cd /
  
−
## Get a list of the installed software ##
+
# Get a list of the installed software
−
dpkg --get-selections > $LOCAL_FOLDER/installed-software.$DOW
+
dpkg --get-selections > $FOLDER/installed-software-$DOW.txt
  
−
## Backup the server config files ##
+
## PostgreSQL database (Needs a /root/.pgpass file)
 +
which -a psql
 +
if [ $? == 0 ] ; then
 +
    echo "SQL dump of PostgreSQL databases"
 +
    su - postgres -c "pg_dump --inserts dspace > /tmp/dspace-db.sql"
 +
    cp /tmp/dspace-db.sql $FOLDER/dspace-db-$DOW.sql
 +
    su - postgres -c "vacuumdb --analyze dspace > /dev/null 2>&1"
 +
fi
 +
 
 +
# Backup MySQL database (Needs a /root/.my.cnf file)
 +
which -a mysql
 +
if [ $? == 0 ] ; then
 +
    echo "SQL dump of MySQL databases"
 +
    mysqldump -A > $FOLDER/mysql-db-$DOW.sql
 +
fi
 +
 
 +
# Backup '/etc' folder
 
echo "Archive '/etc' folder"
 
echo "Archive '/etc' folder"
−
tar czf $LOCAL_FOLDER/etc.tgz.$DOW etc/
+
tar czf $FOLDER/etc-$DOW.tgz etc/
  
−
## Backup the '/root' folder ##
+
# Backup '/root' folder
 
echo "Archive '/root' folder"
 
echo "Archive '/root' folder"
−
tar czf $LOCAL_FOLDER/root.tgz root/
+
tar czf $FOLDER/root.tgz root/
  
−
## Backup the '/usr/local' folder which houses customised software ##
+
# Backup '/usr/local' folder
 
echo "Archive '/usr/local' folder"
 
echo "Archive '/usr/local' folder"
−
tar czf $LOCAL_FOLDER/usr-local.tgz usr/local/
+
tar czf $FOLDER/usr-local.tgz usr/local/
  
−
## Backup the Dspace postgres database which houses the catalog of the digital assets ##
+
# View backup folder
−
su - postgres -c "pg_dump dspace > /tmp/dspace-db.sql"
+
echo ""
−
cp /tmp/dspace-db.sql $LOCAL_FOLDER/dspace-db.sql-$DOW
+
echo "** Backup folder **"
−
su - postgres -c "vacuumdb --analyze dspace > /dev/null 2>&1"
+
ls -lhS $FOLDER
  
−
## View the backup folder ##
+
# Show disk usage
−
ls -lhS $LOCAL_FOLDER
+
echo ""
 +
echo "** Disk usage **"
 +
df -h
  
−
## Timestamp the end of the backup ##
 
 
TIME=`date`
 
TIME=`date`
−
echo "Backup for $LOCAL_SERVER ended: $TIME"
+
echo "Backup ended: $TIME"
−
} > $BACKUP_LOGFILE
 
  
−
## Make a daily copy of the backup log file ##
+
} > $FILE
−
cp $BACKUP_LOGFILE $BACKUP_LOGFILE.$DOW
 
  
−
## Email the backup logfile to the root user ##
+
# Prepare email
−
cat $BACKUP_LOGFILE.$DOW | mail -s "Daily backup log from $HOSTNAME" root
+
cat $FILE | mail -s "BACKUP : $DOW : $SERVER" %email-address%
  
 
### EOF ###
 
### EOF ###
Line 78: Line 109:
 
  sudo /usr/local/bin/backup.sh
 
  sudo /usr/local/bin/backup.sh
 
Then check the files in the backup folder by typing the following:
 
Then check the files in the backup folder by typing the following:
−
  sudo ls -lh /var/backup
+
  sudo ls -lh /home/backup
  
 
===Setup Daily Backup Job===
 
===Setup Daily Backup Job===
 
A cron job entry is added to the root crontab to run the script at midnight each day. To add the script to the root crontab type the following '''as the root user''':
 
A cron job entry is added to the root crontab to run the script at midnight each day. To add the script to the root crontab type the following '''as the root user''':
 
  sudo crontab -e
 
  sudo crontab -e
−
 
+
Type the following to run backups at midnight:
 
  @midnight /usr/local/bin/backup.sh
 
  @midnight /usr/local/bin/backup.sh
 +
''Or alternatively at 05h00 in the morning in order to reduce system load at midnight.''
 +
00 5 * * * /usr/local/bin/backup.sh
 
{{NANO}}
 
{{NANO}}
  
−
{{CONSOLE}}
+
===References===
−
 
+
====Crontab====
−
'''[[SUNScholar/Disaster Recovery/Backups/Client Setup|Back to Client Setup]]'''
+
*http://www.crontab-generator.org
 +
*http://en.wikipedia.org/wiki/Cron
 +
*http://adminschoice.com/crontab-quick-reference
 +
*http://www.yourownlinux.com/2014/04/schedule-your-jobs-in-linux-with-cron-examples-and-tutorial.html
 +
====Backup/Restore PostgreSQL DB====
 +
*http://www.thegeekstuff.com/2009/01/how-to-backup-and-restore-postgres-database-using-pg_dump-and-psql
 +
*https://www.commandprompt.com/blog/a_better_backup_with_postgresql_using_pg_dump
 +
*http://postgresguide.com/utilities/backup-restore.html
 +
*http://www.enterprisedb.com/resources-community/tutorials-quickstarts/linux/how-use-pgdump-and-pgrestore-postgres-plus-tutorial-
 +
*https://www.pgadmin.org
 +
[[Category:System Administration]]

Latest revision as of 08:17, 22 July 2016

Back to Client Setup

First click here to create a PostgreSQL credentials file.

Create local backup script

Here is a sample script to make local backups which are then sent to the remote backup server.

Type the following. 

sudo nano /usr/local/bin/backup.sh

Now copy and paste the following into the open editor and modify the backup variables to suit your location and server.

  1. Replace %hostname% with the hostname of your server.
  2. Replace %email-address% with the email address of the person responsible for server backups.
#!/bin/bash

# Setup shell to use for backups
SHELL=/bin/bash

# Setup name of local server to be backed up
SERVER="%hostname%"

# Setup event stamps
DOW=`date +%a`
TIME=`date`

# Setup paths
FOLDER="/home/backup"
FILE="/var/log/backup-$DOW.log"

# Do the backups
{
echo "Backup started: $TIME"

# Make the backup folder if it does not exist
if test ! -d /home/backup
then
  mkdir -p /home/backup
  echo "New backup folder created"
  else
  echo ""
fi

# Make sure we're in / since backups are relative to that
cd /

# Get a list of the installed software
dpkg --get-selections > $FOLDER/installed-software-$DOW.txt

## PostgreSQL database (Needs a /root/.pgpass file)
which -a psql
if [ $? == 0 ] ; then 
    echo "SQL dump of PostgreSQL databases"
    su - postgres -c "pg_dump --inserts dspace > /tmp/dspace-db.sql"
    cp /tmp/dspace-db.sql $FOLDER/dspace-db-$DOW.sql
    su - postgres -c "vacuumdb --analyze dspace > /dev/null 2>&1"
fi

# Backup MySQL database (Needs a /root/.my.cnf file)
which -a mysql
if [ $? == 0 ] ; then 
    echo "SQL dump of MySQL databases"
    mysqldump -A > $FOLDER/mysql-db-$DOW.sql
fi

# Backup '/etc' folder 
echo "Archive '/etc' folder"
tar czf $FOLDER/etc-$DOW.tgz etc/

# Backup '/root' folder
echo "Archive '/root' folder"
tar czf $FOLDER/root.tgz root/

# Backup '/usr/local' folder
echo "Archive '/usr/local' folder"
tar czf $FOLDER/usr-local.tgz usr/local/

# View backup folder
echo ""
echo "** Backup folder **"
ls -lhS $FOLDER

# Show disk usage
echo ""
echo "** Disk usage **"
df -h

TIME=`date`
echo "Backup ended: $TIME"

} > $FILE

# Prepare email
cat $FILE | mail -s "BACKUP : $DOW : $SERVER" %email-address%

### EOF ###
NANO Editor Help
CTL+O = Save the file and then press Enter
CTL+X = Exit "nano"
CTL+K = Delete line
CTL+U = Undelete line
CTL+W = Search for %%string%%
CTL+\ = Search for %%string%% and replace with $$string$$
CTL+C = Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)

Now we make the backup script executable. 

sudo chmod 0755  /usr/local/bin/backup.sh

Invoke Local Manual Backup

After you have completed the above, you can start a backup anytime by typing the following as the root user: 

sudo /usr/local/bin/backup.sh

Then check the files in the backup folder by typing the following: 

sudo ls -lh /home/backup

Setup Daily Backup Job

A cron job entry is added to the root crontab to run the script at midnight each day. To add the script to the root crontab type the following as the root user: 

sudo crontab -e

Type the following to run backups at midnight: 

@midnight /usr/local/bin/backup.sh

Or alternatively at 05h00 in the morning in order to reduce system load at midnight. 

00 5 * * * /usr/local/bin/backup.sh
NANO Editor Help
CTL+O = Save the file and then press Enter
CTL+X = Exit "nano"
CTL+K = Delete line
CTL+U = Undelete line
CTL+W = Search for %%string%%
CTL+\ = Search for %%string%% and replace with $$string$$
CTL+C = Show line numbers

More info = http://en.wikipedia.org/wiki/Nano_(text_editor)

References

Crontab

Backup/Restore PostgreSQL DB

Retrieved from "http://wiki.lib.sun.ac.za/index.php?title=SUNScholar/Disaster_Recovery/Backups/Local&oldid=44096"