Showing posts with label UNIX. Show all posts
Showing posts with label UNIX. Show all posts

Tuesday, December 1, 2009

Managing and Archiving Log Files Using Find and Tar

You could probably write a script and put this in the cron scheduler...  Here is the quick and dirty manual way to do it.

I wanted a way to archive everything in a particular directory and remove files (and only files) in that directory.  I was running out of space on one of my filesystems and I knew "log" files were eating up most of my space.  I still had plenty of space on my /tmp filesystem.  I wanted to get rid of files older than 15 days after I archived them so here is what I did.

I created a place for my archive files.
# mkdir -p /tmp/logs

I then changed directories to the place the log files lived.
# cd /(path to log files)/

I then, as a sanity check, ran this command to make sure it included the files I wanted and excluded those I didn't.
# find . -mtime +15 | xargs ls -l

After I validated this list was the files I wanted to archive, I ran the following command to tar and compress the files.
# find . -mtime +15 | xargs tar cvfz /tmp/logs/111609logs.tar.gz

This will tar up and compress everything older than 15 days.  I named my log file using a date stamp 15 days in the past. Now I want to remove files older than 15 days, but only files, not directories.  I used this command.
# find . -type f -mtime +15 | xargs rm -f

The "find" command will find everything so using the "-type f" option it will only find files.

I was able to free up about 3GB.  I hope this helps others out there.

Monday, November 30, 2009

Copying Directory Structures Between Linux or UNIX Systems

As I build or rebuild servers that need a particular directory structure, I wanted a fast way to replicate or reproduce the empty directory structure with the appropriate owners and permissions.  Historically, I would have used the tar command for this and only selected directories.  There is another, and in my opinion more efficient, way to do this using the cpio command.

I wanted to recreate the following two directory structures (including all sub-directories, which there are many of):
/u02/prn/app
/u02/prn/oradata

On the server you want to model the directory structure(s) after you would run the following commands.
# find /u02/prn/app -type d | cpio -ov >/tmp/appdirs.cpio
# find /u02/prn/oradata -type d | cpio -ov >/tmp/oradatadirs.cpio

Now using "scp" I can copy these files to the server I want to recreate the directory structure on.
# scp /tmp/*.cpio destination_server_here:/tmp/.

Now I will extract the archive which will create the directory structures I want (with the appropriate owners and permissions).  Note:  run this command from the "/" filesystem.  If you run it from /tmp, it will look for u02 in tmp.  It obviously doesn't live there.  So, do it from /
# cpio -iv </tmp/appdirs.cpio
# cpio -iv </tmp/oradatadirs.cpio

Done!  Not quite magic, but almost.

Assumptions:  You have the same users and groups created on the destination server as you do on the source server.

Hope this helps someone out there.