Tuesday, December 1, 2009

Managing and Archiving Log Files Using Find and Tar

You could probably write a script and put this in the cron scheduler...  Here is the quick and dirty manual way to do it.

I wanted a way to archive everything in a particular directory and remove files (and only files) in that directory.  I was running out of space on one of my filesystems and I knew "log" files were eating up most of my space.  I still had plenty of space on my /tmp filesystem.  I wanted to get rid of files older than 15 days after I archived them so here is what I did.

I created a place for my archive files.
# mkdir -p /tmp/logs

I then changed directories to the place the log files lived.
# cd /(path to log files)/

I then, as a sanity check, ran this command to make sure it included the files I wanted and excluded those I didn't.
# find . -mtime +15 | xargs ls -l

After I validated this list was the files I wanted to archive, I ran the following command to tar and compress the files.
# find . -mtime +15 | xargs tar cvfz /tmp/logs/111609logs.tar.gz

This will tar up and compress everything older than 15 days.  I named my log file using a date stamp 15 days in the past. Now I want to remove files older than 15 days, but only files, not directories.  I used this command.
# find . -type f -mtime +15 | xargs rm -f

The "find" command will find everything so using the "-type f" option it will only find files.

I was able to free up about 3GB.  I hope this helps others out there.

Monday, November 30, 2009

Copying Directory Structures Between Linux or UNIX Systems

As I build or rebuild servers that need a particular directory structure, I wanted a fast way to replicate or reproduce the empty directory structure with the appropriate owners and permissions.  Historically, I would have used the tar command for this and only selected directories.  There is another, and in my opinion more efficient, way to do this using the cpio command.

I wanted to recreate the following two directory structures (including all sub-directories, which there are many of):
/u02/prn/app
/u02/prn/oradata

On the server you want to model the directory structure(s) after you would run the following commands.
# find /u02/prn/app -type d | cpio -ov >/tmp/appdirs.cpio
# find /u02/prn/oradata -type d | cpio -ov >/tmp/oradatadirs.cpio

Now using "scp" I can copy these files to the server I want to recreate the directory structure on.
# scp /tmp/*.cpio destination_server_here:/tmp/.

Now I will extract the archive which will create the directory structures I want (with the appropriate owners and permissions).  Note:  run this command from the "/" filesystem.  If you run it from /tmp, it will look for u02 in tmp.  It obviously doesn't live there.  So, do it from /
# cpio -iv </tmp/appdirs.cpio
# cpio -iv </tmp/oradatadirs.cpio

Done!  Not quite magic, but almost.

Assumptions:  You have the same users and groups created on the destination server as you do on the source server.

Hope this helps someone out there.

Friday, November 13, 2009

System Monitoring - Cacti Virtual Appliance and Virtual Box

Cacti Virtual Appliance and Virtual Box
I wanted a fast and easy solution to do some monitoring on my network.  I have used mrtg and rddtool before, and it has served my needs in the past.  This time I wanted something that was move feature packed and robust.  Enter Cacti.  It didn’t seem too hard to set this up from scratch, but I really did not want to spend a lot of time messing around with it.  I found http://www.virtualappliances.net/ had a pre-built Cacti Virtual Appliance for VMware in OVF format (Note:  I just noticed on 8/16/2010 that this is not a valid site anymore - so the rest of this post is invalid unless you can find the VA-Cacti.ovf file).  Well Virtual Box can import the OVF format.   So here is what I did:

1.    Download the files from http://www.virtualappliances.net/downloads/esx/i386/VA-Cacti/
2.    Note…  when I downloaded the VA-Cacti.ovf file it wanted to save it with an xml extension.  I let it, and then once it was saved on my hard drive I changed the extension from xml to ovf.
3.    Open VirtualBox
4.    Select File and Import Appliance
5.    Click on Choose and navigate to the OVF File.  Select it and click next.
6.    Unselect/uncheck the NIC.  Click the import button.
7.    The VA-Cacti virtual machine should show up in your list.  Highlight it and click on Settings.
8.    Click System, uncheck floppy and cdrom
9.    Click Display, change Video Memory from 4 to 7.
10.    Click Network, Enable the Network Adapter.  I selected the Intel PRO/1000 MT Desktop NIC and Selected Bridged Adapter.
11.    Click OK
12.    For whatever reason,  VM-ware tools is installed and messes up the /etc/fstab file.  Plus we don’t need or want vmware tools…  We are using Virtual Box.  There is probably a cleaner way to do this, but a quick and dirty way to overcome this issue is…
13.    Start the Appliance but push ESC right away.  Select the “recovery mode” option from the menu.
14.    The default root password is root.
15.    Enter it to get in to maintenance mode.
16.    Change directories to /etc
17.    Remove the vmware tools directory.
# rm -R vmware-tools
18.    Change directories to /etc/rc2.d
19.    Remove the S19vmware-tools startup/shutdown script
# rm S19vmware-tools
20.    reboot
21.    As the system is shutting down, you may get a warning about not being able to find vmware-tools…  So what…
22.    The system will reboot.
23.    Take note of the server’s address.  You will see a message stating VA Management Console can be access at https://:8000
24.    Open a browser and go to that address.
25.    Log in using the default admin account.  admin/admin
26.    Click on Configuration at the top and select time keeping.  Change the date/time/zone to match your area.
27.    Click save and you should get a message stating that the changes will take affect the next time you restart.
28.    If you want, you can set a static IP and other config settings from this page, but for the purposes of this example we will leave everything else allow.
29.    Restart the server now by click on the reboot button at the top of the web page.
30.    When the server comes back up, verify the IP address.
31.    Now open a browser and go to http://
32.    Click on Next.  Click Next again.  Click Finish.
33.    You will now be presented with a Cacti login screen. The default username and password is admin/admin.  Once you enter that you will be forced to change your password.  For my test, I just entered admin in twice to leave it unchanged.  Bad security?  Yes, but this is just my Proof of Concept test.
34.    Now you are in and the system is up.  Before we go any farther, I am going to point you to the Cacti user manual.  This is a must read.  Cacti, is very configurable and in being so, it is easy to render your system useless and broken.  The manual is here:  http://docs.cacti.net/manual:087
35.    Assumptions:  Cisco router with an SNMP RO community string set.
36.    With that out of the way, I will go through one example of monitoring a router and setting up graphs for it.
37.    Click create devices and click add in the upper right hand corner.
38.    Fill in a description,  enter the IP Address or DNS name, select Cisco Router from the Host Template drop down.  Select SNMP Version 2 and enter the appropriate RO community string.  Then click create.  If successful, you should see at the top of your screen “Save Successful” and under Ping Results… “Host is alive”.
39.    To the right of that a “Create Graphs for this Host” link.  Click it.
40.    Towards the top, check the box for “Create: Cisco - CPU Usage” then scroll down and click create.  Click create again.
41.    Now select the interface(s) you want to monitor and click create.
42.    Now click on the left hand side toward the top “Graph Trees”.  Then click “Default Tree”
43.    Then click add.  The Parent Item is root, the Tree Item Type is Host, Choose your host from the drop down and select create.
44.    Now click the graphs tab at the top.  You should see your host under the default tree.  It will take approximately 5 to 10 minutes for graphs to be created and enough data to start plotting.
Have Fun!


Friday, October 30, 2009

Converting a Physical Server (or Virtual Server) to Oracle VM (or Xen)

I needed to move some physical Windows 2003 servers.  The site I wanted to move these servers to already had a substantial Oracle VM Server environment that I had built up.  Oracle VM is based off of Xen.  Using Oracle's P2V whitepaper led me to a converted virtual machine that would not boot.  After much searching I found a process that worked for me.  Pulling from different areas on the web the process I came up with is this.

Note:  This process works for Physical servers or Virtual servers (Example:  VMWare).

   1. On the server you wish to convert you need to do some prep work.  You'll need to make sure the server has ide device drivers available and the registry needs to be made aware.  Believe it or not Microsoft has a KB article that details what we need to do.  The section of this KB article we need is under the heading "More Information".  The article can be found here - http://support.microsoft.com/kb/314082
   2. In summary you need to extract the Atapi.sys, Intelide.sys, Pciide.sys, and Pciidex.sys files from the %SystemRoot%\Driver Cache\I386\Driver.cab file and copy the files to the %SystemRoot%\System32\Drivers folder and then copy the registry section to a file (Example: mergeide.reg) and merge the file in to the registry.
   3. Use the free VMWare converter tool to convert the Physical machine to a vmdk file.  If you are doing this on a server (HP or Dell for example) make sure you unselect any management partitions.
   4. Copy this vmdk file to your Oracle VM (or Xen) Server and run the following command:  # qemu-img convert (filename).vmdk (filename).img
   5. Then create a Windows 2003 guest VM (or which ever the OS is of the box you are converting) and make sure it is shutdown.
   6. Remove the img file for the guest you just created and replace it with the one you created in step 4.  Make sure you rename your new image file to the same name as the one you removed in the beginning of step 6.
   7. The guest VM should boot up.  Once your image is up, remove any VMWare tools or conversion programs and remove any Vendor specific management programs (example HP or Dell utilities).


Done.  This process worked for me.  Hope it will help you too.

Wednesday, October 7, 2009

Virtual File Systems: For Fun, Security, and Profit

Assumptions:  Your OS is Linux.
Do you need to bring up an FTP or NFS server on an existing system but want to keep things separated? Do you wanted to have a portable file system you can take with you? Do you want to "hide" things but are not worried about encryption? With Linux it is possible and easy. Here’s how.

Creating a virtual file system.

Assumptions: You have a large file system with plenty of free space. For this example, we’ll assume this file system is mounted on /home and there is well over 20GB of free space available.

Make a file in your home directory called portable.ext3 (you can call it whatever you want).
# touch /home/bob/portable.ext3

Now, let’s make this file a 16GB “empty” file. What you say? That’s right. We are going to take the file portable.ext3 and make is 16GB is size but completely empty. How?
# dd if=/dev/zero of=/home/bob/portable.ext3 bs=16G count=1
So we are creating a file with a 16GB block size at a count of 1. So… 1 block, 16GB in size.

We now have an “empty” 16GB file. Let’s format it.

Wait a minute. It is a file. You can’t format it.

Yes we can.
# mkfs.ext3 /home/bob/portable.ext3
You may get a warning, but it can be ignored.

So now we have a 16GB file that has been formatted as an ext3 filesystem.

What is left to do? Mount it.
Create an empty directory so you can mount the filesystem.
# mkdir -p /home/bob/mntdir (you can call the directory whatever you want).

Now let’s mount it.
# mount –o loop,rw /home/bob/portable.ext3 /home/bob/mntdir

Ta-Da. Do a “df -h” and you should see your “new” 16GB filesystem.

What can you do with it? You can share it out using NFS. Have it as your FTP root. Or (assuming you are going to unmount it first) copy it to a large enough flash drive and take it to a different linux system. If you are not going to move it between systems, and it will only live on one box, you may want to edit your /etc/fstab so that it will be mounted the next time the system reboots.

Friday, September 25, 2009

Linux Authentication Integration with Active Directory

Note:  Since I originally posted this, Likewise Open came out with version 6.  The instructions below work for version 5.3 and 5.4.

I manage a few dozen Oracle Enterprise Linux (OEL) servers. The timeline to implement these servers and bring them in to production was extremely short. I wanted to have these servers integrated with Active Directory for user authentication. We didn’t have the time to implement any of Oracle's Single Sign On solutions. I also didn’t have the time to properly setup LDAP/WinBind/Samba, so I opted to create a user setup script which would insure the UID would be the same on all the servers. The script worked great. User setup was a snap. The downside was passwords. I set everyone’s password to a default password and set it to expire immediately. Once a user would log in, they would be prompted to change their password. Doesn’t seem like a big deal, but there are close to 40 servers that a user would need to do this on. Not ideal for a long term solution.

So, now that these servers are running in production, I needed to go back and implement a better way to manage user accounts in Linux.

I needed to test my implementation on a non-production server. I installed VirtualBox on my Windows 7 PC and installed OEL as a virtual machine. Perfect. Now I have a box I can break without affecting anyone else.

Next, I did some reading on various ways to authenticate Active Directory accounts on a Linux server. I found a Microsoft TechNet article that looked promising. From what I read, LDAP and LADP/Kerberos were probably not the best solutions. The author suggested Winbind. The article walked me through the steps. I was able to join the server to the domain. Wow, that was pretty easy. Now the true test; logging in. I could not log in to the server from the GUI. I could, however, SSH to the server and log in as my Active Directory persona. I checked log files, tried a couple dozen different things and got nowhere. Maybe the fix was easy and I just missed it. In either case, after spending about two (work) days trying to get it to work, I needed to find a different/easier solution.

Enter LikeWise Open.

Right from their website, LikeWise Open
• Joins non-Windows systems to Active Directory domains in a single step from the command line or from a GUI
• Authenticates users with a single user name and password on both Windows and non-Windows
• Enforces the same password policies for non-Windows users and Windows users
• Supports multiple forests with one-way and two-way cross forest trusts
• Caches credentials in case your domain controller goes down
• Provides single sign-on for SSH and Putty
• Next-generation authentication engine that supports Kerberos, NTLM, and SPNEGO
• No schema changes to Active Directory required

Instructions for install:
1. Download the installer package and make it executable.
2. Double click the package and install.
3. Join the server to the AD domain (/opt/likewise/bin/domainjoin-cli join domainName joinAccount).
4. Reboot
5. Login as an AD user (domain\username) from GUI or (domain\\username) from the command line or SSH.

Yes. It was that easy. It worked.

I wanted to make sure that a specific Active Directory group had “sudo” privileges, so I (as root) ran visudo to edit the sudoers file. The line added looked like this.
%MYDOMAIN\\Info^Services ALL=(ALL) ALL

If your group name has a space in it use the ^ charterer in place of a space.
Example: Info Services would be Info^Services
Note the double \ or \\. On the command line “\” is interpreted as an escape charterer so you need two of them.

I tried to sudo as me and it worked!

I change the group on a directory to an Active Directory group.
Example: # chgrp MYDOMAIN\\Info^Services /tmp/testdir

Then I did a: # ls -l /tmp

There it was. The group on the directory was MyDomain\Info^Services

Slick.

I still need to do a little research to make sure this solution meets my security requirements, but so far I am impressed.

P.S. In no way am I associated with the company LikeWise. I like the solution and thought others may too.

Wednesday, September 23, 2009

Imaging Your PC or Server

I have been using an open source tool called G4L for a while to create images of my PCs and Servers.  I figured I would share my process to help others looking to get images of their Windows or Linux Servers or PCs.  Hope this helps some of you out there. Of course, do this at your own risk. If you are not careful you can really mess things up.

Creating a server Image:

Prerequisites:
1. Laptop on the same network as the device to be imaged
2. FTP server on the Laptop capable of supporting files over 4GB.
FileZilla recommended.
http://filezilla-project.org/download.php?type=server
3. USB Harddrive connected to the laptop with two directories: img and g4l (by default g4l will look for and create images in the /img directory)
4. Get the g4l scripts (files3.tar.gz). File can be obtained at: http://sourceforge.net/projects/g4l/files/
5. Unzip the zip file to the g4l directory on the USB drive.

Assumptions:
1. You are using a CDROM ISO that is appropriate for your hardware platform architecture.

Windows or Linux - Use the G4L ISO. If you run in to issues regarding drivers, etc use Knoppix or System Rescue ISO.
     http://sourceforge.net/projects/systemrescuecd/files/sysresccd-x86/1.3.0/
     http://sourceforge.net/projects/g4l/files/g4l%20ISO%20images/g4l-v0.30.iso/g4l-v0.30.iso/download
     http://knopper.net/knoppix-mirrors/index-en.html
2. Server can be booted from CDROM Drive
3. Server can see the network after being booted from CDROM
4. FTP Server is started on the laptop, anonymous access granted, read/write/delete access granted, and the laptop is connected to the network.
5. Root directory of your FTP Server is your external USB hard drive attached to the laptop.

Instructions:

1. Boot Server from CDROM. G4L-ISO, Knoppix or System Rescue CD
1a.  If you are using the G4L-ISO you can skip down to step 17 assuming you have a DHCP server handing out an IP address.
2. Set IP Address if one was not assigned via DHCP (ifconfig eth0 10.10.10.75 netmask 255.255.255.0) - use your own IP and netmask here.
3. Test network connectivity by pinging the FTP server
4. Create directory /tmp/g4l (mkdir -p /tmp/g4l)
5. Change directories to /tmp/g4l (cd /tmp/g4l)
6. FTP to the FTP Server (ftp 10.10.10.17) - Use the IP of your FTP server.
7. Log in (anonymous)
8. Change to the g4l directory (cd g4l)
9. Turn on binary mode (bin)
10. Turn on hash mode (hash)
11. Turn off prompt (prompt)
12. Get the g4l scripts (mget *)
13. quit
14. Change directories to / (cd /)
15. export PATH=$PATH:/tmp/g4l/
16. chmod 777 /tmp/g4l/*
17. Run G4L (/g4l)
18. Answer the following prompts: Yes, OK, OK
19. Config option D with the appropriate IP Address of the FTP Server
20. Config option E - Set the username and password (anonymous:foo)
21. Config option F - Set the name of the file you will be creating
22. Select Option H
23. Select the partition/drive you want to image
24. Confirm image process is under way
25. Wait.
26. When complete, you can exit the g4l utility and halt the system.

Restoring a Server Image:

Prerequisites:
Same as above

Assumptions:
Same as above AND
1. The drive you will restore the image to is the same size or larger than the size of the drive you took the image from.
2. You know that by restoring your image to a drive of larger size will not automagically make your partition larger. If you restore to a larger drive you will need to use another tool (like gparted live) to change the size of your partitions.
3. You are fully aware that any data that existed on the drives prior to doing the restore will be wiped out. Make sure you restore an image to new drives, drives that contain no data, or drives you don't mind being wiped out.

Instructions:
Follow the same steps above until you get to #21
1. Here for option F, instead of putting in a name of an image to create, you will select the name of the image already created.
2. Instead of selecting option H to do a backup, select option I to restore.