Me in IT UNIX/Linux Consultancy is based in Utrecht, The Netherlands and specialized in UNIX and Linux consultancy. Experience with Red Hat Enterprise Linux (Red Hat Certified Architect), Fedora Project, CentOS, OpenBSD and related released Open Source products makes Me in IT UNIX/Linux Consultancy a great partner in implementing, maintaining and upgrading your environment.

Open Source software is an important aspect of any Linux distribution. Me in IT UNIX/Linux Consultancy tries to use Open Source software where possible and tries to share experiences actively. In the articles section you will find many UNIX/Linux adventures shared for others to benefit.

Apache Tomcat 7 spec file RPM

I tried to find an RPM for Apache Tomcat version 7, but could not find one. You can use this one, it requires the source code, downloadable from Apache Tomcat's website, under "Source Code Distributions".

This SPEC file creates an RPM "apache-tomcat" that installs in /opt/tomcat, and the default web applications (apache-tomcat-manager, apache-tomcat-ROOT, apache-tomcat-docs, apache-tomcat-examples, apache-tomcat-host-manager). An init-script is included at the bottom, that needs to be available in the SOURCES directory, named "apache-tomcat-iniscript".

Downloads:
SOURCE:
Apache Tomcat SRC rpm

x86_64:
Apache Tomcat 86_64 rpm
Apache Tomcat ROOT application 86_64 rpm
Apache Tomcat docs 86_64 rpm
Apache Tomcat example application 86_64 rpm
Apache Tomcat host manager application 86_64 rpm
Apache Tomcat manager application 86_64 rpm

So far it's been fine, but any comments would be appreciated.

apache-tomcat.spec:

Name: apache-tomcat
Version: 7.0.20
Release: 1
Summary: Open source software implementation of the Java Servlet and JavaServer Pages technologies.
Group: Productivity/Networking/Web/Servers
License: Apache Software License.
Url: http://tomcat.apache.org
Source: %{name}-%{version}-src.tar.gz

BuildRoot: %{_tmppath}/%{name}-%{version}-build
BuildRequires: ant
BuildRequires: ant-trax
Requires: java-1.6.0-openjdk
BuildArch: x86_64

%description
Apache Tomcat is an open source software implementation of the Java Servlet and JavaServer Pages technologies. The Java Servlet and JavaServer Pages specifications are developed under the Java Community Process.

%package manager
Summary: The management web application of Apache Tomcat.
Group: System Environmnet/Applications
Requires: %{name} = %{version}-%{release}
BuildArch: noarch

%description manager
The management web application of Apache Tomcat.

%package ROOT
Summary: The ROOT web application of Apache Tomcat.
Group: System Environmnet/Applications
Requires: %{name}-%{version}-%{release}
BuildArch: noarch

%description ROOT
The ROOT web application of Apache Tomcat.

%package docs
Summary: The docs web application of Apache Tomcat.
Group: System Environmnet/Applications
Requires: %{name}-%{version}-%{release}
BuildArch: noarch

%description docs
The docs web application of Apache Tomcat.

%package examples
Summary: The examples web application of Apache Tomcat.
Group: System Environmnet/Applications
Requires: %{name}-%{version}-%{release}
BuildArch: noarch

%description examples
The examples web application of Apache Tomcat.

%package host-manager
Summary: The host-manager web application of Apache Tomcat.
Group: System Environmnet/Applications
Requires: %{name}-%{version}-%{release}
BuildArch: noarch

%description host-manager
The host-manager web application of Apache Tomcat.

%prep

%setup -q -n %{name}-%{version}-src
# This tells ant to install software in a specific directory.
cat << EOF >> build.properties
base.path=%{buildroot}/opt/apache-tomcat
EOF

%build
ant

%install
rm -Rf %{buildroot}
mkdir -p %{buildroot}/opt/apache-tomcat
mkdir -p %{buildroot}/opt/apache-tomcat/pid
mkdir -p %{buildroot}/opt/apache-tomcat/webapps
mkdir -p %{buildroot}/etc/init.d/
mkdir -p %{buildroot}/var/run/apache-tomcat
%{__cp} -Rip ./output/build/{bin,conf,lib,logs,temp,webapps} %{buildroot}/opt/apache-tomcat
%{__cp} %{_sourcedir}/apache-tomcat-initscript %{buildroot}/etc/init.d/apache-tomcat

%clean
rm -rf %{buildroot}

%pre
getent group tomcat > /dev/null || groupadd -r tomcat
getent passwd tomcat > /dev/null || useradd -r -g tomcat tomcat

%post
chkconfig --add %{name}

%preun
if [ "$1" = "0" ] ; then
service %{name} stop > /dev/null 2>&1
chkconfig --del %{name}
fi

%files
%defattr(-,tomcat,tomcat,-)
%dir /opt/apache-tomcat
%config /opt/apache-tomcat/conf/*
/opt/apache-tomcat/bin
/opt/apache-tomcat/lib
/opt/apache-tomcat/logs
/opt/apache-tomcat/temp
/opt/apache-tomcat/pid
%dir /opt/apache-tomcat/webapps
/var/run/apache-tomcat
%attr(0755,root,root) /etc/init.d/apache-tomcat

%files manager
/opt/apache-tomcat/webapps/manager

%files ROOT
/opt/apache-tomcat/webapps/ROOT

%files docs
/opt/apache-tomcat/webapps/docs

%files examples
/opt/apache-tomcat/webapps/examples

%files host-manager
/opt/apache-tomcat/webapps/host-manager

%changelog
* Fri Aug 19 2011 - robert (at) meinit.nl
- Updated to apache tomcat 7.0.20
- Split (example) applications into their own RPM.
* Mon Jul 4 2011 - robert (at) meinit.nl
- Initial release.

apache-tomcat-initscript:

#!/bin/sh
#
# apache-tomcat
#
# chkconfig: - 85 15
# description: Jakarta Tomcat Java Servlets and JSP server
# processname: java
# pidfile: /var/run/apache-tomcat/pid

. /etc/rc.d/init.d/functions

# Set Tomcat environment.
USER=tomcat
LOCKFILE=/var/lock/apache-tomcat
export BASEDIR=/opt/apache-tomcat
export TOMCAT_HOME=$BASEDIR
export CATALINA_PID=/var/run/apache-tomcat/pid
export CATALINA_OPTS="-DHOME=$BASEDIR/home -Xmx512m -Djava.awt.headless=true"

case "$1" in
  start)
        echo -n "Starting apache-tomcat: "
        status -p $CATALINA_PID apache-tomcat > /dev/null && failure || (su -p -s /bin/sh $USER -c "$TOMCAT_HOME/bin/catalina.sh start" > /dev/null && (touch $LOCKFILE ; success))
        echo
        ;;
  stop)
        echo -n "Shutting down apache-tomcat: "
        status -p $CATALINA_PID apache-tomcat > /dev/null && su -p -s /bin/sh $USER -c "$TOMCAT_HOME/bin/catalina.sh stop" > /dev/null && (rm -f $LOCKFILE ; success) || failure
        echo
        ;;
  restart)
        $0 stop
        $0 start
        ;;
  condrestart)
       [ -e $LOCKFILE ] && $0 restart
       ;;
  status)
        status -p $CATALINA_PID apache-tomcat
        ;;
  *)
        echo "Usage: $0 {start|stop|restart|condrestart|status}"
        exit 1
        ;;
esac

service status somedaemon - status: unrecognized service

I created an initscript for a daemon recently it works well for starting and stopping, but "service status somedaemon" would return "status: unrecognized service"

After a bit of googling I found that the top few lines of the initscript are critical. It should be a describe here:

#!/bin/sh
#
# somedaemon
#
# chkconfig: - 85 15
# description: Some daemon
# processname: somedaemon
# pidfile: /var/run/somedaemon.pid

Here is what these lines do.

  1. #!/bin/sh - This is a shell script.
  2. # somedaemon - This is the name of the service used with for "service status somedaemon". If this is missing, you'll see: "status: unrecognized service".
  3. chkconfig: - 85 15 - This tells chkconfig three things, firstly (the dash) at what runlevel to start it. A dash means default. Secondly at what priority to start and lastly at what priority to stop.
  4. description: Some daemon - Not used.
  5. processname: somedaemon - Not used.
  6. pidfile: /var/run/somedaemon.pid - Used in /etc/rc.d/init.d/functions to determine if the process is running or not.

Linux interview questions

From time to time you might need to interview somebody for a Linux role. It's hard to think of good questions; you don't want to scare somebody with your questions, but you do want to know if the person is knowledgable.

Questions for a junior.

"People that use your Linux server complain it's slow. What tools would you use to check resource usage?"
top, sar, netstat -an, iostat, free, df.

"You discover a disk is full on your Linux server. What do you use to discover where the bigest files/directories are?"

  • To create a short report on the largest directories: du -sk * | sort -nr | head -n10
  • You might use "find" to find large files: find / -size +100M

"A system needs more disk space. You want to add a partition /var/log. You add a drive to the machine, it becomes /dev/sdb. What actions do you take to use this disk?"
Depending if you use LVM or not. If you don't use LVM:

  1. "fdisk /dev/sdb" to add one partition so /dev/sdb1 becomes available.
  2. "mkfs /dev/sdb1" to create a filesystem on it.
  3. "mount /dev/sdb1 /var/log". Actually the data needs to be copied into the new /var/log.

If you do use LVM:

  1. "fdisk /dev/sdb" to add one partition so /dev/sdb1 becomes available.
  2. "pvcreate /dev/sdb1" to make it an LVM device.
  3. "vgextend SomeVolumeGroup /dev/sdb1" to extend an existing Volume Group.
  4. "lvcreate -L 1G -n SomeLogicalVolumeName SomeVolumeGroup" to create a new Logical Volume.
  5. "mkfs /dev/SomeVolumeGroup/SomeLogicalVolumeName" to create a filesystem on it.
  6. "mount /dev/SomeVolumeGroup/SomeLogicalVolumeName /var/log". Actually the data needs to be copied into the new /var/log.

Questions for a medior.

"People that use your Linux server complain it's slow. You've seen the disk usage is high. What can you do to improve this?"

  • Move away services, devide them over different server.
  • Add more memory so disk caching can be used better.
  • Look into the application, why is it reading/writing so much.
  • User faster disks.

"What happens in relation to DNS, SMTP and IMAP when I send send an email to [email protected]?"
My computer is likely to be configure to send the email to a mail server on port 25, SMTP protocol. That mailserver will query the DNS for the MX records of example.com. The mailserver that show up with the lowest priority will be contacted to deliver the email. That mailserver at example.com can accept the email and put in into the imap folder for the user or alias of [email protected]

"You need to setup 100 Red Hat Enterprise Linux systems. If you don't want to walk around and eject and insert a boot CD 100 times, what options would you have?"
Kickstarting would help out. Install one machine as you like it, save /root/anaconda-ks.cfg to a webserver. Setup a PXE (DHCP, TFTP, HTTP, DNS) environment and use that kickstart file to install the rest.

Questions for a senior.

"You have destroyed /etc/pam.d/system-auth and can't login anymore. Another machine has a propper version of /etc/pam.d/system-auth. How would you fix that broken machine?"
The machine needs to be booted in single user mode so you don't get a login prompt. After that here are some option:

  • Start network, use "nc" to get and replace that file.
  • Mount a CD that has the package to fix the broken file.
  • Manually repair it.

"You have installed apache, php and mysql and a webapplication such as Drupal. The webapplication tries to send emails to an external mailserver but fails. What could be the cause when these items have been verified:

  • The web application is correctly configured to use the external mailserver.
  • It's possible to connect to the mailserver on the specified port from the command line.
  • The logs of the mailserver don't record anything when the web application tries to send email."
  • IPtables allows connections out to port 25.

SELinux could be blocking apache from using port 25 on an external system. The logfile /var/log/messages might inform you about it. To fix it issue "setsebool -P httpd_can_network_connect=1".

"What determines the load (w, uptime) of a system?"
The number of processes that are waiting for execution. These processes are in the run queue. Processes could be waiting for io, network or memory allocation.

Drupal on Amazon's Elastic Compute Cloud (EC2)

Hosting Drupal site in the Amazon EC2 cloud is not difficult. Here is a recipe I have used, first attempt was a Fedora 14 EC2 ami, but Fedora 14 comes with php 5.3, which can't be combined with Drupal 5.x. If you only have Drupal 6 (or Drupal 7) sites to host, you can use Fedora 14. If you want to use a "small" instance, please read this bug about readdir64_r. The fix for that bug is easy:

echo "hwcap 1 nosegneg" > /etc/ld.so.conf.d/libc6-xen.conf

Let's continue with Drupal on CentOS. Rightscale provides perfect CentOS amis that can be used on Amazons EC2 platform. If you install one, these are the steps I took to make it Drupal 5, Drupal 6 and Drupal 7 ready:

# Update the software.
yum -y update

# Set the timezone for this machine.
cp /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime

# This images came with 10Gb of ESB storage, I added another 64 Gb volume, here it's called "/dev/sdc".
# Use LVM to be ready to grow in the future.
pvcreate /dev/sdc
vgcreate vg0 /dev/sdc
lvcreate vg0 -L 32G -n var-www
lvcreate vg0 -L 1G -n var-lib-mysql
lvcreate vg0 -L 2G -n root

# Put filesystems on the logical volumes.
mke2fs -j /dev/vg0/var-www
mke2fs -j /dev/vg0/var-lib-mysql
mke2fs -j /dev/vg0/root

# Add the mountpoints to fstab.
echo "/dev/vg0/var-www /var/www/virtualhosts ext3 defaults 0 0" >> /etc/fstab
echo "/dev/vg0/var-lib-mysql /var/lib/mysql ext3 defaults 0 0" >> /etc/fstab
echo "/dev/vg0/root /root ext3 defaults 0 0" >> /etc/fstab

# Create the mountpoints.
mkdir /var/www /var/lib/mysql /root

# Mount all mounpoints in /etc/fstab.
mount -a

# Install the webserver.
yum -y install httpd
service httpd start
chkconfig httpd on

# Install the database server.
yum -y install mysql-server
service mysqld start
chkconfig mysqld on
/usr/bin/mysqladmin -u root password 'YourPassWord'

# Install PHP and all required Drupal php modules.
yum -y install php php-mysql php-mcrypt php-xml php-mbstring php-gd

# Add a single new file in the ESB root filesystem, that includes configurations from /var/www/conf.d
# Using this trick allows you to easily remount the volume on another host in case of troubles.
echo 'Include /var/www/conf.d/*.conf' >> /etc/httpd/conf.d/virtualhosts.conf
mkdir /var/www/conf.d

# Rightscale CentOS images comes with postfix and sendmail. Postfix is enabled, but sendmail is fine for me.
# First erase postfix.
yum -y erase postfix

# Now reinstall sendmail to fix a few permissions.
yum -y reinstall sendmail
service sendmail start
chkconfig on sendmail

# Reboot the box to make sure it's working properly.
reboot

Fedora 14 to be released

So it's not so long before Fedora 14 is released. Lets take a look at some features Fedora 14 will bring that look interesting to me.

  • An Amazon EC2 Image (AIM) for Fedora will be released! Finally move on from that old Fedora 8 image that was available on Amazon EC2.
  • For Desktop Virtualization, Spice is released. This is a KVM feature that enable super fast access to virtualized desktop systesm, Windows, Linux or any other.
  • Perl, Python and Ruby will be updated, nice but not very important to me.
  • OpenSCAP can ensure security compliance. Should be helpful to many customers.

All in all, a pretty good release although it's kind of hard to improve an already good distribution.

Many of the changes done in Fedora, will end up in RHEL. Current estimate is that RHEL 7 will be based on Fedora 16 (earliest) up to Fedora 19. (latest)

Moving a single Drupal instalation into a multisite environment.

If you'd like to move a single installation of Drupal into a multisite environment, use these steps, and replace example.com for your websites name. In this example, Drupal is installed in /var/www/drupal/ .

1. Change the webserver configuration.

This one is obvious, the website was first pointing to an individual installation of Drupal, it needs to be directed to the multi-site installation of Drupal.

2. Copy templates, settings.php and files into the multisite environment.

mkdir -p /var/www/drupal/sites/example.com/
cd /var/www/example.com
cp -Rip html/sites/all/themes html/sites/default/settings.php html/sites/default/files/ /var/www/drupal/sites/example.com/

3. Update the MySQL database with the new paths.

If you have used files (including images) on your website, the path needs to be updated. Earlier files were located on "sites/default/files/", but this will become "sites/example.com/files"

mysql -u root -pPaSsWoRd
USE examplecom;
UPDATE files SET filepath=REPLACE (filepath, 'default', 'example.com');
UPDATE files SET filepath=REPLACE ('filepath', 'images', 'sites/example.com/images') WHERE filepath REGEXP '^images.*';
QUIT;

4. Change the location of the icons for the selected theme.

Go to Administer - Site building - Themes - Your Theme "configure" and change the path to reflect the right one. Mostly this means changing the word "files/" to "sites/example.com/files/".

5. Restart the web server and clean up the old environment.

For Apache, that would be:

apachectl configtest
apachectl restart

Check the website, everything should work, maybe you have to reselect your template to make it look better. If all works well, remove the old code.

rm -R /var/www/example.com

Shrinking a filesystem with LVM

After an installation you might find some file systems are too large, they are almost empty. When you want to use that space for another file system, here are the steps you can take:
Imagine /opt is now 10 Gb, but 1 Gb would be sufficient.

  1. Check if the file system is in use. Using lsof /opt you will get a list of processes that currently use /opt. Stop these processes.
  2. Find out what device is used for /opt with df -h /opt or mount. In my example, I found /dev/mapper/VolGroup/opt hold files on /opt.
  3. Unmount the filesystem, using umount /opt
  4. Resize the filesystem using resize2fs /dev/mapper/VolGroup/opt 1G. This frees the "right" part of the disk that LVM will un-allocate in a moment. All data from the file system is on the "left hand side".
  5. Run lvreduce -L 1G /dev/mapper/VolGroup-opt to shrink the logical volume. (It might warn you that you need to run e2fsck -f /dev/mapper/VolGroup-opt before you can continue.
  6. Remount the filesystem with a command as mount /opt.

For /opt or any other filesystem that can easily be freed from open file handles, the above procedure works fine, but for "busy" filesystems, like /, /var, /usr, and so on, you'd have boot the machine without mounting filesystems. One way to do this is using the installation CD and starting up the "rescue" environment.

VirtualBox fencing and Red Hat Enterprise Linux Cluster Suite

I know there is already a perl script to fence VirtualBox machines, but since I am not very familiar with Perl, a shell scripts seemed easier.

So; when you run VirtualBox and would like (virtual) cluster nodes to be able to fence, you may use this shell script on the virtual machines:

#!/bin/sh -x

# Script to stop and start a virtual machine.
# The only required argument is machinename.

eval $(cat -)

# I use Apple Mac OS X, but any OS may be used.
vboxmanage="/Applications/VirtualBox.app/Contents/MacOS/VBoxManage"

usage () {
/bin/echo "Usage: $0 -a NAME [-o ACTION]"
/bin/echo
/bin/echo " -a NAME"
/bin/echo "   The name of the virtual machine to be fenced."
/bin/echo "   In case it contains spaces, use double quotes."
/bin/echo " -o ACTION"
/bin/echo "   What to do; start|stop|reboot(default)."
/bin/echo
exit 1
}

while [ "$#" -gt 0 ] ; do
case "$1" in
  -a)
   if [ "$2" ] ; then
    vm="$2"
    shift ; shift
   else
    /bin/echo "Missing value for $1."
    /bin/echo
    usage
    shift
   fi
  ;;
  -o)
   if [ "$2" ] ; then
    action="$2"
    shift ; shift
   else
    /bin/echo "Missing value for $1."
    /bin/echo
    usage
    shift
   fi
  ;;
  *)
   /bin/echo "Not a know option, $1."
   usage
   shift
  ;;
esac
done

if [ ! "${action}" ] ; then
action=reboot
fi

if [ ! "${vm}" ] ; then
/bin/echo "Error, please specify a name."
usage
fi

check() {
ssh $host "$vboxmanage showvminfo ${vm}" > /dev/null 2>&1
if [ ${?} != 0 ] ; then
  /bin/echo "Error, VM ${vm} not found, choose one of these:"
  ssh $host "$vboxmanage list vms | sed 's/" .*/"/'"
  exit 1
fi
}


stop() {
ssh $host "$vboxmanage controlvm ${vm} poweroff > /dev/null 2>&1"
}

start() {
ssh $host "$vboxmanage startvm ${vm} > /dev/null 2>&1"
}

reboot() {
stop
sleep 3
start
}

case $action in
start)
  check
  start
;;
stop)
  check
  stop
;;
reboot)
  check
  reboot
;;
*)
  /bin/echo "Unknown action: $action"
  /bin/echo
  usage
;;
esac

To use this script, add this block to every "clusternode" in /etc/cluster/cluster.conf:

    <fence>
        <method name="1">
            <device host="[email protected]" name="vbox" vm="ClusterMember2"/>
        </method>
    </fence>

And create a shared fence device in /etc/cluster/cluster.conf:

    <fencedevices>
        <fencedevice agent="fence_vbox" name="vbox"/>
    </fencedevices>

The important variables are "host" and "vm". "host" is used to connect to the physical box running VirtualBox, "vm" is the actual name of the virtual machine, as displayed by VirtualBox.

Forwarding one port to another

Here is a very simple stick to forward a TCP port from your local workstation to another host. Can be easy to use for debugging purposes:

mkfifo pipe ; cat pipe | nc -l 8080 | nc google.com 80 > pipe ; rm pipe

What this one does:
1) Create a fifo (First in First out) file. This is a very simple type of file, you can put stuff in there with an output redirect (>) and get stuff out there with cat for example. It acts as a temporary buffer.
2) Open that newly created pipe. Anything that gets in, will be printed. (and forwarded in this example to "nc")
3) Open a listening port on your local workstation, listening on port 8080.
4) Open a connection to google.com, on port 80.
5) Send al the output to the earlier created pipe.
6) Remove the pipe when done.

Have a look the netcat homepage, it's a great tool!

Creating an RPM of some binary

We've covered this topic before in this story about creating an RPM from a shell script, but this information might help you better understand how to create an RPM.

So; you've found a piece of software that has no RPM? (Or; your manager tells you to install a piece of software that the development department created.)

Normally you'd use ./configure ; make ; make install, here is how to put that all in an RPM.

Prepare your rpm building environment: (DO THIS AS A USER!)

$ sudo yum install rpm-build
$ mkdir -p RPMBUILD/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
$ echo "%_topdir /home/username/RPMBUILD" >> .rpmmacros

Now copy the software into that newly create structure.

$ cp software.tar.gz RPMBUILD/SOURCES/

And now create a "spec file" for the software. This basically explains rpmbuild how to make the software and what to put in the RPM. This is the most "tweakable" step and might require quite some time to get right. Put this into /home/username/RPMBUILD/SPECS/software.spec:

Name: software
Version: 0.23
Release: 1
Summary: Custom software to run enterprise servers.

Group: Applications/Internet
License: GPLv2
URL: http://meinit.nl/
Source0: %{name}-%{version}.tar.gz
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root

%description
This software runs all enterprise software as a daemon. It's been developed by Me in IT consultancy.

%prep
%setup -q

%build
make

%install
mkdir -p $RPM_BUILD_ROOT/usr/local/bin
install software $RPM_BUILD_ROOT/usr/local/bin/software

%files
%defattr(-,root,root)
%doc README
/usr/local/bin/software

%clean
rm -rf $RPM_BUILD_ROOT

%changelog
* Tue Jun 15 2010 Robert de Bock <[email protected]> - 0.23-1
- Initial build

Good to know; the %install refers to the temporary environment that rpm will create when building this RPM. The %files section refers to what will end up in the RPM. They should correspond; you can't %install a whole bunch of files and only include a few in the $files part. (rpmbuild will display the missing files.

The group can be any line out of /usr/share/doc/rpm-*/GROUPS

So; you are prepared, run this command to so if you got everything correct:

$ rpmbuild -ba software.spec

When it finally builds, you'll find the rpm in /home/username/RPMBUILD/RPMS/$arch/software-0.23-1.$arch.rpm

About Consultancy Articles Contact




References Red Hat Certified Architect By Robert de Bock Robert de Bock
Curriculum Vitae By Fred Clausen +31 6 14 39 58 72
By Nelson Manning [email protected]
Syndicate content