Showing posts with label cron. Show all posts
Showing posts with label cron. Show all posts

Tuesday, January 12, 2010

Filesystem Replication Using rsync and SHH

There are many reasons you may need to replicate a file system.  My reason was for DR purposes.  In a previous post I set up Passwordless SSH sessions between two systems.  This is a requirement if you want to sync file systems on an automated schedule.  I looked on the web for a script that would do what I wanted and I could not find something that met my needs.  So I wrote the script below.  To give credit where credit is due, I borrowed from some ideas and code from Randal K. Michael, author of Mastering UNIX Shell Scripting.  I placed the script below in /usr/local/bin directory on the "source" node and called it fsrsync.bash.  This script will replicated a designated filesystem from a source node to two different nodes after confirming they are "alive".  I used rsync because after the initial sync, future replications are much faster since only update and/or changes are sent and not the entire filesystem.

Here is the script below.  Highlight the contents and select copy.
Note:  When you grab the test here and paste it, do a sanity check on the text to verify the formatting has not changed.
Edit/create /usr/local/bin/fsrsync.bash and paste in the contents.
# vi /usr/local/bin/fsrsync.bash

Script starts below.

#!/bin/bash
#
# SCRIPT: fsrsync.bash
# AUTHOR:
# DATE:
# REV:
#
# PURPOSE: This script is used to replicate the
# /somedir/test filesystem from Node A to Node B and C
#
# set -x # Uncomment to debug this script
#
# set -n # Uncomment to check the script.s syntax
#        # without any execution. Do not forget to
#        # recomment this line!
#
##############################################
# DEFINE FILES AND GLOBAL VARIABLES HERE
##############################################

# Define the target machines to copy data to.
# To specify more than one host enclose the
# hostnames in double quotes and put at least
# one space between each hostname
#
# EXAMPLE: MACHINE_LIST="fred yogi booboo"

MACHINE_LIST="nodeB nodeC"

# Capture the shell script file name

THIS_SCRIPT=$(basename $0)

# The FS_PATTERN variable defines the regular expression
# matching the filesystems we want to replicate with rsync.
# Example:  FS_PATTERN="/home"

FS_PATTERN="/somedir/test"

# Query the system for the hostname

THIS_HOST=$(hostname)

##############################################
# BEGINNING OF MAIN
##############################################

# Comfirm the nodes are alive and replicate
# the filesystems.
echo -e "\n####################################################\n"
echo -e "$THIS_SCRIPT started execution $(date)\n"
echo -e "Verifying the node is alive..."

for M in $MACHINE_LIST
do
    echo "Pinging $M..."
    ping -c1 $M >/dev/null 2>&1
    if (( $? != 0 ))
    then
        echo -e "ERROR: $M host is not pingable...cannot continue..."
        echo -e "...EXITING...\n"
        echo -e "####################################################"
        exit 2
    else
        echo -e "$M is alive... Starting rsync process!\n"
        echo -e "Replicating $FS_PATTERN/ from $THIS_HOST to $M\n"
        #The rsync command is all on one line although it doesn't appear so here.  This comment can be removed.
        rsync -aqz --delete -e ssh $FS_PATTERN/ root@$M:$FS_PATTERN
    fi
echo -e "$THIS_SCRIPT finished execution $(date)\n"
echo -e "####################################################"
done

###############################################
# END OF SCRIPT
###############################################

Make sure you make the file executable.
# chmod 754 /usr/local/bin/fsrsync.bash

You can manually run the file by simply running this command as a user with the appropriate rights.
# /usr/local/bin/fsrsync.bash

Do you want to schedule this script to replicate the file system every 5 minutes and log results?  Add the following entry to the crontab.
# crontab -e
0,5,10,15,20,25,30,35,40,45,50,55 * * * *     /usr/local/bin/fsrsync.bash 2>&1 >> /var/log/fsrsync.log


Do you want to rotate your log file (assuming you use logrotate)?  If so create a file in /etc/logrotate.d/ called fsrsync.
# vi /etc/logrotate.d/fsrsync

The contents of the file should look like this:
/var/log/fsrsync.log {
        weekly
        rotate 4
        nocompress
        missingok
}

I hope this helps someone out there.

PS:  If you want to test logrotate without having to wait a week you can do the following.
# /usr/sbin/logrotate -v /etc/logrotate.d/fsrsync
This will give you details on what it will do and rotate the log if needed.  If you want to force a log rotation, do the following.
# /usr/sbin/logrotate -f /etc/logrotate.d/fsrsync