Using "tar" to backup and exclude files in a script.

I needed to create a manual backup script that I could easily modify to backup specific directories and files while excluding others.  I was having a difficult time excluding files and directories without having to type out each exclusion using --exclude=somefile --exclude=some_other_file --exclude=somedirectory.  My "tar" command would have grown quite large and, in my opinion, is not easy to read. So, I decided to use an array to accomplish my goal.

#!/bin/bash # # SCRIPT: manual_backup.bash # AUTHOR: Bob # DATE: 3-1-2011 # R_DATE: 11-14-2012 # REV: 1.3.T (Valid are A, B, D, T, Q, and P) # (For Alpha, Beta, Dev, Test, QA, and Production) # # PLATFORM: Not platform dependent # # PURPOSE: Backup modified configuration files # # # set -n # Uncomment to check script syntax, without execution. # # NOTE: Do not forget to put the # comment back in or # # the shell script will never execute! # set -x # Uncomment to debug this shell script # ########################################################## # DEFINE FILES AND VARIABLES HERE ########################################################## SCRIPTNAME=$(basename $0 .bash) # List files or directories you want backed up FILELIST=" /etc/hosts /etc/fstab /var/log/*" # List files or directories you want excluded from the backup. # This is an array. EXCLUDELIST=(--exclude=/var/log/messages) EXCLUDELIST+=(--exclude=/var/log/lastlog) EXCLUDELIST+=(--exclude=/var/log/mail) ########################################################## # DEFINE FUNCTIONS HERE ########################################################## function get_date_stamp { DATENOW=$(date +'%m-%d-%y') } function get_host_name { HOST=$(hostname) } ########################################################## # BEGINNING OF MAIN ########################################################## get_date_stamp get_host_name tar -czvf /tmp/$HOST.$SCRIPTNAME.$DATENOW.tar.gz "${EXCLUDELIST[@]}" $FILELIST # End of script

Backing Up your Laptop or PC - Easy!

I was using a program called G4L to image my laptop or pc but I have found a much easier way.  It works with Linux or Windows, no installation is needed, allows you to resize partitions, backup systems, access files, and recover lost data.  It is a live CD called "Redo".  Check it out here http://redobackup.org/.  It rocks!

Taken from the site:

Redo Backup and Recovery is so simple that anyone can use it. It is the easiest, most complete disaster recovery solution available. It allows bare-metal restore. Bare metal restore means that even if your hard drive melts or gets completely erased by a virus, you can have a completely-functional system back up and running in as little as 10 minutes.


All your documents and settings will be restored to the exact same state they were in when the last snapshot was taken. Redo Backup and Recovery is a live CD, so it does not matter if you use Windows or Linux. You can use the same tool to backup and restore every machine. And because it is open source released under the GPL, it is completely free for personal and commercial use.

Backing Up a django Web Server

I have a webserver I want to backup weekly and keep 4 weeks of backup history.
The webserver runs CentOS 5.4 and Django.
In the event of a server crash I'd like to be able to recover my website rather easily.  I figure it would be relatively easy to reinstall the OS and install the applications.  Then all I would need to do is restore the configuration files and the database.

Assumptions:
You have django installed and working
$DBDUMPDIR = where ever you want the Database Dump files to go
$BKUPDIR = where ever you want the Backup files to go.  Note:  This backup directory could be a remote server such as an NFS mount or SMB mount.  This is suggested so if this server crashes, your backups would be on another server.
Edit the scripts below to for "your" appropriate directories and file names.



Here are the four files I will need to backup:
*  /etc/httpd/conf/httpd.conf
*  /etc/httpd/conf/mysite.conf
*  /etc/www/django
*  $DBDUMPDIR/data.json

I created a script called dumpfiles.bash and put it in root's bin directory "/root/bin/dumpfiles.bash".
Make sure that permissions are right to be able to execute the file.
chmod 770 /root/bin/dumpfiles.bash

Here is the script:
----- start of script -----
#!/bin/bash
#
# SCRIPT: dumpfiles.bash
# AUTHOR: Bob
# DATE: 07/01/2010
# REV: 1.blah
#
# PURPOSE: This script is used to backup webserver specific data
#
# set -x # Uncomment to debug this script
#
# set -n # Uncomment to check the script.s syntax
#        # without any execution. Do not forget to
#        # recomment this line!
#
####################
# Define Variables #
####################
# Capture the shell script file name
THIS_SCRIPT=$(basename $0)
#
# Define the start time of the script
STARTTIME=`date +%T`
#
#Set Backup and DB Dump Directories
#Change the directories below to match your environment
BKUPDIR=/mnt/backup/webserver
DBDUMPDIR=/root
#
##################
# Increment Backups #
##################
rm -f $BKUPDIR/bkup4.tar.gz
mv -f $BKUPDIR/bkup3.tar.gz $BKUPDIR/bkup4.tar.gz
mv -f $BKUPDIR/bkup2.tar.gz $BKUPDIR/bkup3.tar.gz
mv -f $BKUPDIR/bkup1.tar.gz $BKUPDIR/bkup2.tar.gz
#
###################
# Dump Database #
###################
# export PYTHONPATH for dumpdata script
export PYTHONPATH='/var/www/django':'/var/www/django/apps'
#
#Change Directories to /var/www/django/mysite
cd /var/www/django/mysite
#
#Backup the DJango database and files to a flat file
python manage.py dumpdata > $DBDUMPDIR/data.json
#
#Get out of the /var/www/django/mysite directory
#Let us go home
cd /root
#
###############################
# Backup and compress important files #
###############################
tar cvfz $BKUPDIR/bkup1.tar.gz /etc/httpd/conf/httpd.conf /etc/httpd/conf.d/mysite.conf /var/www/django $DBDUMPDIR/data.json
----- end of script -----

I want to schedule a weekly backup so I'll use root's cron to do this.
Edit root's cron (assuming you are logged in as root).

Enter the following line to root's cron.
0 4 * * 1 /root/bin/dumpfiles.bash

This will run the script every Monday morning at 4:00 AM.
Now each time the script runs bkup1.tar.gz will be created.  If it already exists the old files will be incremented up to the number 4 giving you four weeks of backup files.

In the event of a disaster and you rebuilt a new server, you'd untar the backup files and copy the files back to their appropriate place and you the manage.py script to restore the django database.

Hope this helps someone out there.

Interactive Tape Backups using TAR and Linux

I sometimes want to run an on-demand backup of either a particular directory or file system.  I wrote an interactive script to do this and thought I would share it.
Assumptions:
1.  You have a tape device attached to your Computer or Server.
2.  You know what device your tape drive is.  Example: /dev/st0
3.  You have the mt-st package installed to manage the tape device.
4.  The tape you are using will be overwritten.
5.  You copy the contents of the script below in to a utility like notepad and check the contents.  Then copy from there in to a script called (whatever you want).
6.  Pay close attention to the command that starts like this:  TAPECHK=$(mt  It show up correctly in this post but if you cut and past the script in to notepad the lines do not match.  Edit it so it looks like it does here in the post.
7.  You make the script executable.
8.  I placed the script in /usr/local/sbin but you can put it where ever it make sense to you.

Here is the script:

#!/bin/bash
#
# SCRIPT: Interactive_2_tape.bash
# AUTHOR: Bob
# DATE: 02/24/2010
# REV:
#
# PURPOSE: This script is used to backup files
# from $SOURCE to $TAPEDEV
#
# set -x # Uncomment to debug this script
#
# set -n # Uncomment to check the script.s syntax
#        # without any execution. Do not forget to
#        # recomment this line!
#
####################
# Define Variables #
####################

# Capture the shell script file name
THIS_SCRIPT=$(basename $0)

# Define the start time of the script
STARTTIME=`date +%T`

# Define current directory to return to at end of script
CURRENTDIR=$PWD

# Ask for the source of the backup
echo "What directory or filesystem do you want to backup?"
echo "Type the directory in this format /dir1/dir2 followed by [ENTER]:"
read SOURCE
echo "Using $SOURCE as the source directory you want to backup."

# Ask for tape device
TAPEDEV="/dev/st0"
echo "I assume your tape device is $TAPEDEV"
read -p "Am I correct? yes/no: "
if [ "$REPLY" = "no" ]; then
     echo "What is your tape device? "
     read TAPEDEV
     echo "Using $TAPEDEV as your tape device."
else
     echo "Using $TAPEDEV as your tape device."
fi


################
# Main Section #
################

# Verify there is a tape in the
# drive and rewind the tape
TAPECHK=$(mt -f $TAPEDEV rewind 2>&1 1>/dev/null)
# If there is no tape tell me and exit out
#  If the mt command return any data then there is an error
#  Check the results of the mt command
if [ "$TAPECHK" != "" ]; then
     echo $TAPECHK
     echo "Check to see if there is a tape in the drive or if the device you entered is valid."
     exit
fi

# If we made it here, there is a tape in the drive and it has rewinded.
# Change to the directory to be backed up
cd $SOURCE
echo "Changing to the $SOURCE directory."

# Back up data
tar cvf $TAPEDEV .

# Rewind the tape again
mt -f $TAPEDEV rewind

# Change back to the directory from where you came
cd $CURRENTDIR
echo "Changing back to the directory you started from: $CURRENTDIR"
# Define the end time of this script
ENDTIME=`date +%T`

# Display the start and end time of this script
echo "$THIS_SCRIPT began at $STARTTIME and finished at $ENDTIME"

exit
#################
# End of Script #
#################

I hope this helps others out there trying to do the same thing.