Articles

Adventures in Red Hat Enterprise Linux, CentOS, Fedora, OpenBSD and other open source solutions.

Drupal on Amazon's Elastic Compute Cloud (EC2)

Hosting Drupal site in the Amazon EC2 cloud is not difficult. Here is a recipe I have used, first attempt was a Fedora 14 EC2 ami, but Fedora 14 comes with php 5.3, which can't be combined with Drupal 5.x. If you only have Drupal 6 (or Drupal 7) sites to host, you can use Fedora 14. If you want to use a "small" instance, please read this bug about readdir64_r. The fix for that bug is easy:

echo "hwcap 1 nosegneg" > /etc/ld.so.conf.d/libc6-xen.conf

Let's continue with Drupal on CentOS. Rightscale provides perfect CentOS amis that can be used on Amazons EC2 platform. If you install one, these are the steps I took to make it Drupal 5, Drupal 6 and Drupal 7 ready:

# Update the software.
yum -y update

# Set the timezone for this machine.
cp /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime

# This images came with 10Gb of ESB storage, I added another 64 Gb volume, here it's called "/dev/sdc".
# Use LVM to be ready to grow in the future.
pvcreate /dev/sdc
vgcreate vg0 /dev/sdc
lvcreate vg0 -L 32G -n var-www
lvcreate vg0 -L 1G -n var-lib-mysql
lvcreate vg0 -L 2G -n root

# Put filesystems on the logical volumes.
mke2fs -j /dev/vg0/var-www
mke2fs -j /dev/vg0/var-lib-mysql
mke2fs -j /dev/vg0/root

# Add the mountpoints to fstab.
echo "/dev/vg0/var-www /var/www/virtualhosts ext3 defaults 0 0" >> /etc/fstab
echo "/dev/vg0/var-lib-mysql /var/lib/mysql ext3 defaults 0 0" >> /etc/fstab
echo "/dev/vg0/root /root ext3 defaults 0 0" >> /etc/fstab

# Create the mountpoints.
mkdir /var/www /var/lib/mysql /root

# Mount all mounpoints in /etc/fstab.
mount -a

# Install the webserver.
yum -y install httpd
service httpd start
chkconfig httpd on

# Install the database server.
yum -y install mysql-server
service mysqld start
chkconfig mysqld on
/usr/bin/mysqladmin -u root password 'YourPassWord'

# Install PHP and all required Drupal php modules.
yum -y install php php-mysql php-mcrypt php-xml php-mbstring php-gd

# Add a single new file in the ESB root filesystem, that includes configurations from /var/www/conf.d
# Using this trick allows you to easily remount the volume on another host in case of troubles.
echo 'Include /var/www/conf.d/*.conf' >> /etc/httpd/conf.d/virtualhosts.conf
mkdir /var/www/conf.d

# Rightscale CentOS images comes with postfix and sendmail. Postfix is enabled, but sendmail is fine for me.
# First erase postfix.
yum -y erase postfix

# Now reinstall sendmail to fix a few permissions.
yum -y reinstall sendmail
service sendmail start
chkconfig on sendmail

# Reboot the box to make sure it's working properly.
reboot

Fedora 14 to be released

So it's not so long before Fedora 14 is released. Lets take a look at some features Fedora 14 will bring that look interesting to me.

  • An Amazon EC2 Image (AIM) for Fedora will be released! Finally move on from that old Fedora 8 image that was available on Amazon EC2.
  • For Desktop Virtualization, Spice is released. This is a KVM feature that enable super fast access to virtualized desktop systesm, Windows, Linux or any other.
  • Perl, Python and Ruby will be updated, nice but not very important to me.
  • OpenSCAP can ensure security compliance. Should be helpful to many customers.

All in all, a pretty good release although it's kind of hard to improve an already good distribution.

Many of the changes done in Fedora, will end up in RHEL. Current estimate is that RHEL 7 will be based on Fedora 16 (earliest) up to Fedora 19. (latest)

Moving a single Drupal instalation into a multisite environment.

If you'd like to move a single installation of Drupal into a multisite environment, use these steps, and replace example.com for your websites name. In this example, Drupal is installed in /var/www/drupal/ .

1. Change the webserver configuration.

This one is obvious, the website was first pointing to an individual installation of Drupal, it needs to be directed to the multi-site installation of Drupal.

2. Copy templates, settings.php and files into the multisite environment.

mkdir -p /var/www/drupal/sites/example.com/
cd /var/www/example.com
cp -Rip html/sites/all/themes html/sites/default/settings.php html/sites/default/files/ /var/www/drupal/sites/example.com/

3. Update the MySQL database with the new paths.

If you have used files (including images) on your website, the path needs to be updated. Earlier files were located on "sites/default/files/", but this will become "sites/example.com/files"

mysql -u root -pPaSsWoRd
USE examplecom;
UPDATE files SET filepath=REPLACE (filepath, 'default', 'example.com');
UPDATE files SET filepath=REPLACE ('filepath', 'images', 'sites/example.com/images') WHERE filepath REGEXP '^images.*';
QUIT;

4. Change the location of the icons for the selected theme.

Go to Administer - Site building - Themes - Your Theme "configure" and change the path to reflect the right one. Mostly this means changing the word "files/" to "sites/example.com/files/".

5. Restart the web server and clean up the old environment.

For Apache, that would be:

apachectl configtest
apachectl restart

Check the website, everything should work, maybe you have to reselect your template to make it look better. If all works well, remove the old code.

rm -R /var/www/example.com

Shrinking a filesystem with LVM

After an installation you might find some file systems are too large, they are almost empty. When you want to use that space for another file system, here are the steps you can take:
Imagine /opt is now 10 Gb, but 1 Gb would be sufficient.

  1. Check if the file system is in use. Using lsof /opt you will get a list of processes that currently use /opt. Stop these processes.
  2. Find out what device is used for /opt with df -h /opt or mount. In my example, I found /dev/mapper/VolGroup/opt hold files on /opt.
  3. Unmount the filesystem, using umount /opt
  4. Resize the filesystem using resize2fs /dev/mapper/VolGroup/opt 1G. This frees the "right" part of the disk that LVM will un-allocate in a moment. All data from the file system is on the "left hand side".
  5. Run lvreduce -L 1G /dev/mapper/VolGroup-opt to shrink the logical volume. (It might warn you that you need to run e2fsck -f /dev/mapper/VolGroup-opt before you can continue.
  6. Remount the filesystem with a command as mount /opt.

For /opt or any other filesystem that can easily be freed from open file handles, the above procedure works fine, but for "busy" filesystems, like /, /var, /usr, and so on, you'd have boot the machine without mounting filesystems. One way to do this is using the installation CD and starting up the "rescue" environment.

Forwarding one port to another

Here is a very simple stick to forward a TCP port from your local workstation to another host. Can be easy to use for debugging purposes:

mkfifo pipe ; cat pipe | nc -l 8080 | nc google.com 80 > pipe ; rm pipe

What this one does:
1) Create a fifo (First in First out) file. This is a very simple type of file, you can put stuff in there with an output redirect (>) and get stuff out there with cat for example. It acts as a temporary buffer.
2) Open that newly created pipe. Anything that gets in, will be printed. (and forwarded in this example to "nc")
3) Open a listening port on your local workstation, listening on port 8080.
4) Open a connection to google.com, on port 80.
5) Send al the output to the earlier created pipe.
6) Remove the pipe when done.

Have a look the netcat homepage, it's a great tool!

Creating an RPM of some binary

We've covered this topic before in this story about creating an RPM from a shell script, but this information might help you better understand how to create an RPM.

So; you've found a piece of software that has no RPM? (Or; your manager tells you to install a piece of software that the development department created.)

Normally you'd use ./configure ; make ; make install, here is how to put that all in an RPM.

Prepare your rpm building environment: (DO THIS AS A USER!)

$ sudo yum install rpm-build
$ mkdir -p RPMBUILD/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
$ echo "%_topdir /home/username/RPMBUILD" >> .rpmmacros

Now copy the software into that newly create structure.

$ cp software.tar.gz RPMBUILD/SOURCES/

And now create a "spec file" for the software. This basically explains rpmbuild how to make the software and what to put in the RPM. This is the most "tweakable" step and might require quite some time to get right. Put this into /home/username/RPMBUILD/SPECS/software.spec:

Name: software
Version: 0.23
Release: 1
Summary: Custom software to run enterprise servers.

Group: Applications/Internet
License: GPLv2
URL: http://meinit.nl/
Source0: %{name}-%{version}.tar.gz
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root

%description
This software runs all enterprise software as a daemon. It's been developed by Me in IT consultancy.

%prep
%setup -q

%build
make

%install
mkdir -p $RPM_BUILD_ROOT/usr/local/bin
install software $RPM_BUILD_ROOT/usr/local/bin/software

%files
%defattr(-,root,root)
%doc README
/usr/local/bin/software

%clean
rm -rf $RPM_BUILD_ROOT

%changelog
* Tue Jun 15 2010 Robert de Bock <[email protected]> - 0.23-1
- Initial build

Good to know; the %install refers to the temporary environment that rpm will create when building this RPM. The %files section refers to what will end up in the RPM. They should correspond; you can't %install a whole bunch of files and only include a few in the $files part. (rpmbuild will display the missing files.

The group can be any line out of /usr/share/doc/rpm-*/GROUPS

So; you are prepared, run this command to so if you got everything correct:

$ rpmbuild -ba software.spec

When it finally builds, you'll find the rpm in /home/username/RPMBUILD/RPMS/$arch/software-0.23-1.$arch.rpm

Setting up iSCSI (target/server and initiator/client) on RHEL

It's quite easy to setup an iSCSI environment on Red Hat Enterprise Linux. Try this easy setup to get a better understanding of iSCSI.

Ingredients:

  1. Two (virtual) machines, a server 172.16.0.1 and a client 172.16.0.2.
  2. Access to the "RHEL Cluster-Storage" channel on Red Hat Network.

N.B. SELinux must be disabled when using this recipe, iptables tcp port 3260 must be opened on the server.

On the server execute these commands to setup a 100 Mb iSCSI target. This target can later be mounted on the client(s).

# yum install scsi-target-utils
# cat /etc/tgt/targets.conf
<target iqn.2010-04.nl.meinit:node1.target1>
backing-store /iscsi1.img
initiator-address 172.16.0.2
</target>
# dd if=/dev/zero of=/iscsi1.img bs=1024 count=102400
# chkconfig tgtd on
# service tgtd start

Now on (all) client(s) follow these steps. (Please pay attention that only one client was give access in the configuration example above; 172.16.0.2.)

# yum install iscsi-initiator-utils

Start iscsi daemon.

# service iscsi start

To see what IQNs are available, run:

# iscsiadm -m discovery -t sendtargets -p 172.16.0.1

The result is a list of IQN(s) available. This discovery is a mandatory step of connecting to the iSCSI target.

Login to the iSCSI target:

# iscsiadm -m node -T iqn.2010-04.nl.meinit:node1.target1 -p 172.16.0.1 -l

If that all works, you have new SCSI devices available, check dmesg and start iscsi at boot time:

# chkconfig iscsi on

In this example the iSCSI target does not have a filesystem. Create it on the client and mount it at boot time:

# fdisk /dev/sda
# mkfs.ext3 /dev/sda1
# echo "/dev/sda1 /mnt ext3 defaults,_netdev 0 0" >> /etc/fstab

You are done, but these commands are quite useful when connecting to an unknown iSCSI device.

To see more about the IQN:

# iscsiadm -m node -T iqn.2010-04.nl.meinit:node1.target1 -p 172.16.0.1

Using "recording" in VI instead of being annoyed

Here is a trick you can use in VI if you need to repeat an action multiple times. Imagine you have this file:

hello
Workd
hello
Workd
hello
Workd

You want to change it to read "Hello world!" Here is what you can do in VI:

  1. Go to the first occurrence of what you would like to have changed.
  2. Enter "q" to initiate a recording.
  3. Enter a letter or digit to save the recording under, like "a" or "1".
  4. Do the actions you would like to repeat.
  5. Enter "q" again to stop and save the recording.
  6. Go to the line where you would like to start running the saved recording/macro.
  7. Enter "@" followed by the letter of digit you have saved the recording under, like "a" or "1".

So, now you have learned what this (annoying) "recording" option can be used for!

Debugging an SSL connection

Imagine you would like to debug an SSL connection to a box, for example HTTPS. Using telnet or netcat, you are not able to do so, but openssl makes this possible, here is how:

First; connect to a machine:

$ openssl s_client -connect mail.google.com:443

You will be given the certificate details:

CONNECTED(00000003)
depth=2 /C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
verify return:1
depth=1 /C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
verify return:1
depth=0 /C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
verify return:1
---
Certificate chain
0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
  i:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
1 s:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
  i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDIjCCAougAwIBAgIQHxn23jXdY6FCkYrVLMCrEjANBgkqhkiG9w0BAQUFADBM
MQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkg
THRkLjEWMBQGA1UEAxMNVGhhd3RlIFNHQyBDQTAeFw0wOTEyMTgwMDAwMDBaFw0x
MTEyMTgyMzU5NTlaMGkxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh
MRYwFAYDVQQHFA1Nb3VudGFpbiBWaWV3MRMwEQYDVQQKFApHb29nbGUgSW5jMRgw
FgYDVQQDFA9tYWlsLmdvb2dsZS5jb20wgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJ
AoGBANknyBHye+RFyUa2Y3WDsXd+F0GJgDjxRSegPNnoqABL2QfQut7t9CymrNwn
E+wMwaaZF0LmjSfSgRSwS4L6ssXQuyBZYiijlrVh9nbBbUbS/brGDz3RyXeaWDP2
BnYyrVFfKV9u+BKLrebFCDmzQ0OpW5Ed1+PPUd91WY6NgKtTAgMBAAGjgecwgeQw
DAYDVR0TAQH/BAIwADA2BgNVHR8ELzAtMCugKaAnhiVodHRwOi8vY3JsLnRoYXd0
ZS5jb20vVGhhd3RlU0dDQ0EuY3JsMCgGA1UdJQQhMB8GCCsGAQUFBwMBBggrBgEF
BQcDAgYJYIZIAYb4QgQBMHIGCCsGAQUFBwEBBGYwZDAiBggrBgEFBQcwAYYWaHR0
cDovL29jc3AudGhhd3RlLmNvbTA+BggrBgEFBQcwAoYyaHR0cDovL3d3dy50aGF3
dGUuY29tL3JlcG9zaXRvcnkvVGhhd3RlX1NHQ19DQS5jcnQwDQYJKoZIhvcNAQEF
BQADgYEAicju7fexy+yRP2drx57Tcqo+BElR1CiHNZ1nhPmS9QSZaudDA8jy25IP
VWvjEgaq13Hro0Hg32ZNVK53qcXwjWtnCAReojvNwj6/x1Ciq5B6D7E6eiYDSfXJ
8/a2vR5IbgY89nq+wuHaA6vspH6vNR848xO3z1PQ7BrIjnYQ1A0=
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
issuer=/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
---
No client certificate CA names sent
---
SSL handshake has read 1778 bytes and written 343 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
   Protocol  : TLSv1
   Cipher    : AES256-SHA
   Session-ID: 53530BBF94619E255B7956A18D9B9F26241B2A1BF16F30C18C73C88A60200E5F
   Session-ID-ctx:
   Master-Key:
B40F4E1D533F88AF9248E6576CA4E4CFC2C4BD092816DB7EF9D4FE650EA62B4CAC1F23C36892866B40E3502E67D52CF1
   Key-Arg   : None
   Krb5 Principal: None
   Start Time: 1264674178
   Timeout   : 300 (sec)
   Verify return code: 0 (ok)
---

Now you can enter commands in plain text, just as you would using telnet or netcat:

HEAD / HTTP/1.0

Here is what you get:

HTTP/1.0 302 Found
Cache-Control: private
Location: http://www.google.com
Content-Type: text/html; charset=UTF-8
Content-Length: 218
Date: Thu, 28 Jan 2010 10:23:05 GMT
Server: GFE/2.0
X-XSS-Protection: 0

read:errno=0

An easy trick that can also be used to connect to SSL-ed STMP connections, IMAPS, POP3S, etc.

Making an RPM for a shell script.

So, you have written an enterprise quality shell script and would like to deploy it on serveral Red Hat based machines? Creating an RPM will make this easy to do. Here are the steps required.

1. Install rpmbuild so you may start to build your own RPMs.
2. Package your shell script into a tar.gz file and move that to /usr/src/redhat/SOURCES/

# tar -cvzf shell-script-0.1.tar.gz shell-script-0.1
# mv shell-script-0.1.tar.gz /usr/src/redhat/SOURCES/

3. Create a .spec file that describes where everything is.
# cat /usr/src/redhat/SPECS/shell-script.spec
Summary: The do it all script. (Enterprise quality)
Name: shell-script
Version: 0.1
Release: 1
URL:     http://meinit.nl
License: GPL
Group: Applications/Internet
BuildRoot: %{_tmppath}/%{name}-root
Requires: bash
Source0: shell-script-%{version}.tar.gz
BuildArch: noarch

%description
A shell script.

%prep
%setup

%build

%install
rm -rf ${RPM_BUILD_ROOT}
mkdir -p ${RPM_BUILD_ROOT}/usr/bin
install -m 755 shell-script.sh ${RPM_BUILD_ROOT}%{_bindir}

%clean
rm -rf ${RPM_BUILD_ROOT}

%files
%defattr(-,root,root)
%attr(755,root,root) %{_bindir}/shell-script.sh

%changelog
* Tue Jan 12 2010 Robert de Bock <[email protected]>
- Uberscript!

3. Build it!
# rpmbuild --bb /usr/src/redhat/SPECS/shell-script.spec

4. Install it!
# rpm -Uvh /usr/src/redhat/RPMS/noarch/shell-script-0.1.1.noarch.rpm

About Consultancy Articles Contact




References Red Hat Certified Architect By Robert de Bock Robert de Bock
Curriculum Vitae By Fred Clausen +31 6 14 39 58 72
By Nelson Manning [email protected]
Syndicate content