Fred Clausen

Using "recording" in VI instead of being annoyed

Here is a trick you can use in VI if you need to repeat an action multiple times. Imagine you have this file:

hello
Workd
hello
Workd
hello
Workd

You want to change it to read "Hello world!" Here is what you can do in VI:

  1. Go to the first occurrence of what you would like to have changed.
  2. Enter "q" to initiate a recording.
  3. Enter a letter or digit to save the recording under, like "a" or "1".
  4. Do the actions you would like to repeat.
  5. Enter "q" again to stop and save the recording.
  6. Go to the line where you would like to start running the saved recording/macro.
  7. Enter "@" followed by the letter of digit you have saved the recording under, like "a" or "1".

So, now you have learned what this (annoying) "recording" option can be used for!

Debugging an SSL connection

Imagine you would like to debug an SSL connection to a box, for example HTTPS. Using telnet or netcat, you are not able to do so, but openssl makes this possible, here is how:

First; connect to a machine:

$ openssl s_client -connect mail.google.com:443

You will be given the certificate details:

CONNECTED(00000003)
depth=2 /C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
verify return:1
depth=1 /C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
verify return:1
depth=0 /C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
verify return:1
---
Certificate chain
0 s:/C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
  i:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
1 s:/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
  i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDIjCCAougAwIBAgIQHxn23jXdY6FCkYrVLMCrEjANBgkqhkiG9w0BAQUFADBM
MQswCQYDVQQGEwJaQTElMCMGA1UEChMcVGhhd3RlIENvbnN1bHRpbmcgKFB0eSkg
THRkLjEWMBQGA1UEAxMNVGhhd3RlIFNHQyBDQTAeFw0wOTEyMTgwMDAwMDBaFw0x
MTEyMTgyMzU5NTlaMGkxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh
MRYwFAYDVQQHFA1Nb3VudGFpbiBWaWV3MRMwEQYDVQQKFApHb29nbGUgSW5jMRgw
FgYDVQQDFA9tYWlsLmdvb2dsZS5jb20wgZ8wDQYJKoZIhvcNAQEBBQADgY0AMIGJ
AoGBANknyBHye+RFyUa2Y3WDsXd+F0GJgDjxRSegPNnoqABL2QfQut7t9CymrNwn
E+wMwaaZF0LmjSfSgRSwS4L6ssXQuyBZYiijlrVh9nbBbUbS/brGDz3RyXeaWDP2
BnYyrVFfKV9u+BKLrebFCDmzQ0OpW5Ed1+PPUd91WY6NgKtTAgMBAAGjgecwgeQw
DAYDVR0TAQH/BAIwADA2BgNVHR8ELzAtMCugKaAnhiVodHRwOi8vY3JsLnRoYXd0
ZS5jb20vVGhhd3RlU0dDQ0EuY3JsMCgGA1UdJQQhMB8GCCsGAQUFBwMBBggrBgEF
BQcDAgYJYIZIAYb4QgQBMHIGCCsGAQUFBwEBBGYwZDAiBggrBgEFBQcwAYYWaHR0
cDovL29jc3AudGhhd3RlLmNvbTA+BggrBgEFBQcwAoYyaHR0cDovL3d3dy50aGF3
dGUuY29tL3JlcG9zaXRvcnkvVGhhd3RlX1NHQ19DQS5jcnQwDQYJKoZIhvcNAQEF
BQADgYEAicju7fexy+yRP2drx57Tcqo+BElR1CiHNZ1nhPmS9QSZaudDA8jy25IP
VWvjEgaq13Hro0Hg32ZNVK53qcXwjWtnCAReojvNwj6/x1Ciq5B6D7E6eiYDSfXJ
8/a2vR5IbgY89nq+wuHaA6vspH6vNR848xO3z1PQ7BrIjnYQ1A0=
-----END CERTIFICATE-----
subject=/C=US/ST=California/L=Mountain View/O=Google Inc/CN=mail.google.com
issuer=/C=ZA/O=Thawte Consulting (Pty) Ltd./CN=Thawte SGC CA
---
No client certificate CA names sent
---
SSL handshake has read 1778 bytes and written 343 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 1024 bit
Compression: NONE
Expansion: NONE
SSL-Session:
   Protocol  : TLSv1
   Cipher    : AES256-SHA
   Session-ID: 53530BBF94619E255B7956A18D9B9F26241B2A1BF16F30C18C73C88A60200E5F
   Session-ID-ctx:
   Master-Key:
B40F4E1D533F88AF9248E6576CA4E4CFC2C4BD092816DB7EF9D4FE650EA62B4CAC1F23C36892866B40E3502E67D52CF1
   Key-Arg   : None
   Krb5 Principal: None
   Start Time: 1264674178
   Timeout   : 300 (sec)
   Verify return code: 0 (ok)
---

Now you can enter commands in plain text, just as you would using telnet or netcat:

HEAD / HTTP/1.0

Here is what you get:

HTTP/1.0 302 Found
Cache-Control: private
Location: http://www.google.com
Content-Type: text/html; charset=UTF-8
Content-Length: 218
Date: Thu, 28 Jan 2010 10:23:05 GMT
Server: GFE/2.0
X-XSS-Protection: 0

read:errno=0

An easy trick that can also be used to connect to SSL-ed STMP connections, IMAPS, POP3S, etc.

Running an Open Source Phone Part 1

Introduction - Why do we need an Free Software phone?

As we all know, when purchasing a new phone, it often has a slew of restrictions. On the telephony side, you are usually only allowed to use an authorised type of SIM card in the phone but there are far greater restrictions as to what you can do with the software on the phone. Almost all phones out there use proprietary software and, in the cases where a Free Software kernel is used, it is always hidden away and you can't modify your phone by installing a free GNU/Linux distribution on it since only authorised, signed operating systems will be booted.

Besides the software in the phone not being Free (as in freedom) software, there are often arbitrary restrictions on how you can use the phone. For example, DRM to restrict how you can play the music you have legally purchased, you cannot install you own applications but only applications approved by the Telco and/or phone vendor - think of the iPhone.

I am writing a series of articles in order to share my experiences and hopefully help people get familiar with Free Software on mobile devices, specifically cellular telephones. This first article will focus on a high level introduction to some of the hardware and software available for open source/free software phones. For those interested - I am using a Neo Freerunner running QTextended as my daily phone.

The Hardware

All this Free Software is no good if there is no phone you can install it on so I will now give a non-exhaustive list of the mobile devices I know about on which you can install free software.

The Neo Freerunner

The Neo Freerunner is designed to be an open phone from the ground up - the manufacturing diagrams are published as CAD files which anyone can use as a basis for another phone. The Neo Freerunner is the most promising, truly open phone that I am aware of and has a highly active community developing software for the Neo Freerunner and future phones from Openmoko.

The software on this phone (covered in Part 2) is not quite ready for end-user use but can be used as a daily phone by enthusiasts. That said, I expect that basic functionality will be stable in half a year.

Google G1

The Google G1 is a Linux based phone brought out by Google, it seems they want to compete with the Apple iPhone. The retail G1 is a locked phone that will only run authorised images so no change there. However, you can gain access to the full functionality of your phone in two ways; rooting the phone or buying the developer G1 called the "Android Dev Phone 1". Once this has been done you can proceed to experiment to your hearts content with the underlying Linux system making up the G1 - with some limitations.

The Trolltech Greenphone

The now discontinued Green phone used a software stack called Qtopia created by Trolltech (now owned by Nokia) - the same people who develop the QT toolkit in use by such projects as KDE and countless other, smaller applications.

The idea behind the Greenphone was to promote Qtopia as a mobile development platform and not as an end-user telephone. Trolltech no longer ships the Greenphone and the Qtopia software stack as been renamed to QTextended. QTextended has just released version 4.4.3, which will be the last release of the QTextended platform as this too is being discontinued but a community maintained version will still be available and may even become better than the Trolltech version.

Nokia Internet Tablet

The Nokia N810 supports the running of Open Source software - the main software stack target at this device is the Maemo plaform but it also supports QTextended and Debian GNU/Linux. By installing Debian on the N810 you can access to the vast software repositories available to Debian systems.

iPhone

Yes, you read correctly, you can now run Linux on your iPhone. This project is still in its very early stages and already seems to be laying the groundwork quite well. Definitely worth keeping an eye on. Also, I suspect you will need a jail-broken phone in order to install Linux on your iPhone and Apple may release updates to their boot loader ROM that will make it difficult to install Linux on the iPhone.
Netbooks

And, let's not forget, the ever popular Netbooks being made by seemingly all major computer manufacturers. While not strictly speaking a phone or a "tablet", they are nevertheless very mobile and so I will cover them here. The recent Netbook trend all started with the Asus Eee PC which made people realise that they just need "good enough" computing rather than a super computer on their lap.

One can easily install any Linux distribution on these devices and, when combined with a mobile broadband device (aka "dongle"), you have a powerful, mobile Internet device. Especially useful for those of us that are on call!

A very good history of the Netbook can be found at Arstechnica.

Next Article

In the next article I will be taking a deeper look at the various Free Software stacks that are available for running on your mobile device. Stay tuned!

Linux in flight!

I was on the place, about to fly back from a great holiday in South Africa, when they had a problem with a transmitter on the aircraft. So, they had to "reboot" the whole aircraft. While this was happening I got a chance to see the entertainment system boot up. And, too my delight, I saw that it runs Linux!

Writing directly to a USB disk in OS X

Disclaimer : This procedure erases all data on the target volume so use the volume relevant to your setup, eg. that of your USB disk. I and my associates at meinit do not take responsibility if you overwrite the wrong volume and lose your music, movies, thesis, world peace plan etc. And there might be an easier way to do this, I await comments.

Today I would like to share a short note on how to write a raw disk image to a USB memory stick in OS X. The raw disk image can, for example, be a bootable filesystem image for a OS installer. In my case Fedora 8's "diskboot.img".

The main problem is that I was used to Linux's way of device naming but under OS X if you wish to write directly to a disk you need to use the "raw" version of the device. For example /dev/disk1 has raw device /dev/rdisk1.

The other issue I encountered is that when I insert the USB memory stick then /dev/disk1 exists but is mounted so I cannot write to it directly. If I eject the disk then the device node also disappears and I can't write to it. So we need to use the "Disk Utility" to properly unmount the volume, this utility is contained in Applications -> Utilities. Start this up, making sure your USB stick is plugged in. Once started you will see a list of storage devices, much like below :

screenshot of Disk Utility

Here you can see the USB storage quite easily. Highlight the partition within your storage media and click "eject". Now the device node still exists and you can use "dd" to write to the disk directly, /dev/disk1 in my case. My command to write to the disk was :

$ sudo dd if=./diskboot.img of=/dev/disk1 bs=8k

Adjust as required. Good luck and happy new year! :-)

Using Keychain for SSH logins

I would like to make a short intro to Keychain, this utility makes handling SSH keys easier and also saves you having to type in your passphrase every time you SSH to a remote machine using public key authentication.

Most Linux distributions and BSD variants already have Keychain packaged or easily installable in some way. I will leave installing keychain as an exercise to the reader. Also, I recommend following Mr Robert's fine guide on using SSH keys before starting to setup Keychain.

Once you have your SSH keys configured and Keychain installed then using keychain is quite easy. The following steps are required :

  • Initial Test
  • Modify your shell startup scripts to automatically start Keychain

Initial Test

So, our first step is to manually step through the process of feeding Keychain our SSH keys :

keychain@silverado:~$ keychain ~/.ssh/id_rsa

KeyChain 2.6.6; http://www.gentoo.org/proj/en/keychain/ Copyright 2002-2004 Gentoo Foundation; Distributed under the GPL

* Initializing /home/keychain/.keychain/silverado-sh file...
* Initializing /home/keychain/.keychain/silverado-csh file...
* Initializing /home/keychain/.keychain/silverado-fish file...
* Starting ssh-agent
* Adding 1 ssh key(s)...
Enter passphrase for /home/keychain/.ssh/id_rsa:
Identity added: /home/keychain/.ssh/id_rsa (/home/keychain/.ssh/id_rsa)

keychain@silverado:~$

When it asks for your passphrase, enter the passphrase you used during key creation. Once this is successfully completed you need to setup the shell environment. As you can see, Keychain creates many files that contain the required environment variables to make the ssh-agent information accessible across logins and shell instances. You will need to source the appropriate file, depending on your shell. I am using Bash, so I will use the silverado-sh file. Another point is that the file name contains your hostname so if your host name is "mymachine" then the file will be mymachine-sh in the ~/.keychain directory.

The method by which we will acquire the variables stored in the silverado-sh (or whatever it is called on your system) will be by sourcing the files. Like so :

keychain@silverado:~$ source ~/.keychain/silverado-sh

and then to verify the variables are there :

keychain@silverado:~$ env | grep SSH_A
SSH_AGENT_PID=24627
SSH_AUTH_SOCK=/tmp/ssh-EUqFg24626/agent.24626
keychain@silverado:~$

Now try and log into a machine that uses your public key for authentication, you should not be prompted for your passphrase.

Automatically Starting Keychain and Sourcing Files

Now we don't feel like doing that every time so we can put that in our shell initialisation file, in my case ~/.bashrc. Insert the following, replacing "silverado-sh" with your own Keychain environment file :

keychain ~/.ssh/id_rsa
source ~/.keychain/silverado-sh

Tada! Finished. But for the full Keychain treatment I refer you to the Gentoo Documentation.

Very Simple but Secure Database Backup

I am using the very fine DBMail as a MySQL email backup, more on this in other articles. Please note that this is my first article, so please be gentle. But for now, I wanted to share how I backup this email store and hence a database. I have written a script that :

  • Dumps the relevant database.
  • Compresses the dump file.
  • Encrypts the database using Gnu Privacy Guard (but any method will do).
  • Uploads the database to a remote location using SFTP.

The prerequisites for using the script "as is" are :

  • MySQL dump user with sufficient privileges to dump the required database.
  • ~/.my.cnf configured for user having sufficient privileges to backup the DB. For example:

[client]
user = dbuser
password = dbpass
database = dbmail

  • Working GPG installation.
  • Remote backup host running an sftp server.
  • Local copy of sftp OpenSSH client.
  • Keychain for password-less login to remote server. More on keychain in a future article.

Note that this script stems from a quick and dirty script that I wrote for myself so use what is in this article more as a guideline :-) They script itself will require some modification in your environment. And, without further ado, here is the script :

#!/bin/sh
# Author : Fred Clausen <ftclausen@gmail.com>

database="dbmail"
gpg_user="Friedrich"
local_dir="/data/tmp"
remote_host="remote-host.example.com"
remote_user="fred"
remote_dir="/var/tmp"

source $HOME/.keychain/$HOSTNAME-sh
cur_name="$local_dir/$database-`date +%a`.sql.gz"
batch="`mktemp`"

if [ -f ${cur_name}.gpg ]; then
        rm -f ${cur_name}.gpg
fi

mysqldump --databases $database | gzip > $cur_name
if [ $? != 0 ]; then
        echo "Error dumping data"
        exit 1
fi

gpg -r $gpg_user -e $cur_name

cat > $batch <<END
cd $remote_dir
put ${cur_name}.gpg
END

sftp -b $batch $remote_user@$remote_host

rm $batch

You will obviously need to modify the variables at the top to values suitable for your environment. Also, see how this script recycles file names to keep a week's worth of backups.

Good luck!

Syndicate content