Me in IT UNIX/Linux Consultancy is based in Breukelen, The Netherlands and specialized in UNIX and Linux consultancy. Experience with Red Hat Enterprise Linux (Red Hat Certified Architect), Fedora Project, CentOS, OpenBSD and related released Open Source products makes Me in IT UNIX/Linux Consultancy a great partner in implementing, maintaining and upgrading your environment.

Open Source software is an important aspect of any Linux distribution. Me in IT UNIX/Linux Consultancy tries to use Open Source software where possible and tries to share experiences actively. In the articles section you will find many UNIX/Linux adventures shared for others to benefit.

Cygwin is not as scary as it looks

Most people on the windows platform know Cygwin. ("Cygwin is a Linux-like environment for Windows.") As I never use Windows, I feared programs like these, but it turns out cygwin is quite usable. It's even possible to run shell scripts that normally run on my Mac OS X machine.

Download the installer, select all packages you want. (Don't worry, all "generic" tools (ls, cd, grep, awk, ps, bash) are installed by default.) I added "openssh", "netcat", "xterm" and some others. Dependencies will be resolved automatically. The installer downloads everything. You can run the installer again to add extra packages.

You end up with a "Cygwin" icon. Double click it to start your terminal. It's not really a terminal, but looks quite like it.

Some things are strange or missing, like "top" that's missing, permissions (ls -al) look strange, just as the directory structure. But; take some distance from these details and conclude that you have "bash" running on your windows machine!

The command comm is more difficult that you might think

Have you ever used the command comm? It's a Linux command used to compare two (sorted) files. Comm produces three columns of output:
1: Lines only in file 1.
2: Lines only in file 2.
3: Lines in bothe files.

You can surpress columns by using options like "-1", "-2", "-12" and so on.

Imagine file 1 contains:

$ cat file1
A
B
C

And file 2 contains:

$ cat file2
A
C
D

Than these options (left) would produce this output (right):

Option output explanation
-1 ACD Show lines only in file 2 and in both files
-2 ABC Show lines only in file 1 and in both files
-3 BD Show lines in file 1 and in file 2, but not in both files
-12 AC Show lines in both files
-13 D Show lines only in file 2
-23 B Show lines only in file 1
-123 (no output) Surppress all columns

Linux permission system explained

When you are new to Linux or don't use Linux on a daily basis, finding out how file permissions work can be challenging. Here is an as short as possible guide, which can be applied on UNIX, Linux, Mac OS X, FreeBSD, OpenBSD, and other UNIX-like operating systems. We'll call those systems *nix in this guide.

*nix splits permissions in thee groups for files and directories:

  1. Owner This is the person/account that owns the file.
  2. Group This refers to a set of users. 1 or more persons/accounts can be member of a group.
  3. World If you'r not the Owner, and not part of a Group for a specific file or directory, you automatically match World. Normally the permissions given to World, are restricted.

Besides ownership of files and directories, certain permissions can be given as well:

  1. Read The entity (Owner, Group or World) is allowed to read a file.
  2. Write The entity is allowed to write a file, or to create files in the directory for which these permissions are set.
  3. Execute The entity is allowed to execute the file (scripts or binaries) or to change into this directory.

These permissions are set using chmod. (Change Mode.) Ownership of file is altered with chown. (Change Owner)

Chmod wants to know what permissions you give to a file or directory. This value is built up on four fields.

The zeroth field represents the special bits. (Set User id, Set Group id and Stikcy bit, see below.) Most users will not set this bit, which makes it "0" by default, which means: "No special permissions set."
The first field represents the permissions you give to the Owner.
The second field represents the permissions you give to the Group.
The third fiels represents the permissions you give to the World.

Chmod uses numerical arguments to set permissions, to illustrate it a bit: chmod 750 myscript.sh would change permissions for the file myscript.sh.

Read permissions equals a value of 4.
Write permissios equals a value of 2.
Execute permissions equals a value of 1.

Add the numbers representing the permissions you'd like to give to a Owner, Group or World.

So here is a list of common permissions:

  • 750 - The Owner may read, write and execute. The Group may read and execute. (but not write) The world may not do anything with this file.
  • 755 - Same as above, but the world may now also read and execute the file.
  • 700 - The Owner may do everything, the Group and the world may not do anything.
  • 640 - The Owner may read and write, the Group may read and the world may not do anything.
  • 644 - Same as above, but the world may now also read the file.
  • 600 - The Owner may read and write, the rest may not do anything.

Some "weird" permissions, mostly because they are broken or very rare:

  • 000 - Nobody may do anything with this file. Effectively archive the file. Maybe removing the file would be more appropriate.
  • 007 - The Owner and Group can't do anything, but the World can. I can't think of a situation where this would apply.
  • 100 and 300 - The Owner may execute but not read, so the execute will not work.
  • 200 - The owner may write, but not read. The only situation I could imagine is a logfile where some application may write to, but is not allowed to read the file.

There are some special permissions you can give, these permissions go into the zeroth field. You'd use chmod like this to set no special permissions: chmod 0750 myscript.sh.

  1. Sticky bit A value of 1. This bit is commonly set to directories and means that you may only remove file in such a directory that are owned by you. The filesystem /tmp is commonly set with that bit; everyone can write there, but you can only remove/rename files that are owned by yourself.
  2. Set Group id A value of 2. The file is executed as the Group of the file. Some groups have quite some premissions. Imagine /bin/rm (to remove files and directories) having a Set Group id bit, all users in the Group of /bin/rm may remove a whole lot of files. When you are using Set User id or Set Group id bits, you should be very sure you know what you are doing.
  3. Set User id A value of 4. If this permission is given, the program is executed as the Owner of the file. Could be dangerous; think of a PHP script that tries to reboot a machine...

So 4750 would mean the file may be executed by the owner and the group, and will be executed as the owner.

Imagine a script would have 4775 permissions and would be owned by root:users; a user could edit the script, and the world could execute it with roots permission!

Just to remind you once more; Set Group or User id bits are dangerous, know what you are doing when using them!

How to enable tftp server on Mac OS X

When you would like to use TFTP on your Mac OS X machine, take these simple steps:

  1. Change /System/Library/LaunchDaemons/tftp.plist Quite simple; remove two lines <key>Disabled</key> and <true/>. Besides that, I added the -l option with <string>-l</string>, but that's optional.
  2. Load tftp.plist into launchd The command launchdctl load /System/Library/LaunchDaemons/tftp.plist will do that.
  3. Monitor /var/log/system.log With the command tail -f /var/log/system.log

Now place the required files in /private/tftpboot and you are done!

Change ssh configuration when the "Location" changes on Mac OS X

Have you ever used the "Location" facility in Mac OS X? It changes network settings for you when you work from different locations, like "work", "home", "mobile" or any other setting. As far as I can tell it's designed to change:

  • IP, netmask, gateway
  • Used interfaces (airport, ethernet, etc)
  • Proxy server

What's missing here is SSH settings. You configure your ssh-client in ~/.ssh/config manually. In my case I can login a machine from home directly, but when I am somewhere else, I need to use a step-stone.

That would mean I'd have to change ~/.ssh/config every time I am on a different location! Here is a solution to the problem, inspired by an article about Location based scripts.

  1. Create a new Location in Mac OS X Click on the Apple logo in the left corner, Location - Network Preferences - Edit Locations - The + symbol and so on.
  2. Copy/create ssh configurations, call them ~/.ssh.$PROFILE and replace $PROFILE by the name you came up with for you profile names. I use "~/.ssh/config.Automatic" and "~/.ssh/config.Mobile".
  3. Add a few lines to /~.locationchanger These lines actually link ~/.ssh/config to the right ssh configuration.
    location=$(scselect 2>&1|grep ' \* ' | awk '{print $NF}' | sed 's/(//g;s/)//g')
    if [ -f ~/.ssh/config.$location ] ; then
    rm ~/.ssh/config
    ln -s ~/.ssh/config.$location ~/.ssh/config
    fi
  4. Make the ~/.locationchanger executable Use chmod 750 ~/.locationchanger to do that.
  5. Create a file ~/Library/LaunchAgents/LocationChanger.plist The content should be like this:
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
            "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
    <plist version="1.0">
    <dict>
            <key>Label</key>
            <string>tech.inhelsinki.nl.locationchanger</string>
            <key>ProgramArguments</key>
            <array>
                    <string>/Users/robertdb/.locationchanger</string>
            </array>
            <key>WatchPaths</key>
            <array>
                    <string>/Library/Preferences/SystemConfiguration</string>
            </array>
    </dict>
    </plist>
  6. Load that LocationChanger into launchd. Use launchctl load ~/Library/LaunchAgents/LocationChanger.plist to do this, or reboot.

From now on the script ~/.locationchanger will be started when changing Location.

Some items that might be changed when you switch locations are using a stepstone to ssh to machines or using corkscrew to puch through http proxies.

This tool uses scselect, an apple tool that confiures or prints locations.

SCSELECT(8)               BSD System Manager's Manual              SCSELECT(8)

NAME
     scselect -- Select system configuration "location"

SYNOPSIS
     scselect [-n] [new-location-name]

DESCRIPTION
     scselect provides access to the system configuration sets, commonly
     referred to as "locations".  When invoked with no arguments, scselect
     displays the names and associated identifiers for each defined "location"
     and indicates which is currently active.  scselect also allows the user
     to select or change the active "location" by specifying its name or iden-
     tifier.  Changing the "location" causes an immediate system re-configura-
     tion, unless the -n option is supplied.

     At present, the majority of preferences associated with a "location"
     relate to the system's network configuration.

     The command line options are as follows:

     -n  Delay changing the system's "location" until the next system boot (or
         the next time that the system configuration preferences are changed).

     new-location-name
         If not specified, a list of the available "location" names and asso-
         ciated identifiers will be reported on standard output.  If speci-
         fied, this argument is matched with the "location" names and identi-
         fiers and the matching set is activated.

SEE ALSO
     configd(8)

HISTORY
     The scselect command appeared in Mac OS X Public Beta.

Mac OS X                       November 4, 2003                       Mac OS X

Calculating a prime number with a shell script

For experimental purposes I tried to write a shell script that calculates a whole range of prime numbers. The script seems to be working, but is not very efficient. While writing this script I found a regular expression that does the same thing, only a couple of hundred times faster.

Here is that experimental shell script to calculate a prime number:

#!/bin/sh

throughnoother() {
# A function to calculate if the only argument given is not dividable through any other number.
# Set the status to no. No means, not a prime number. When every test is successful, change
# the status to yes.
status=no
# num starts at 1, and is increased by one each time.
num=1
# This for loop consists of numbers up to half the value of the given argument.
for divider in $(while [ $num -lt $(($1/2)) ] ; do num=$(($num+1)) ; echo $num ; done) ; do
  # This is not the test to divide through its own value, so skip that.
  if [ $1 != $divider ] ; then
   # This tests to see if the rounded calculation can be reversed and gives the same result.
   if [ $((($1/$divider)*$divider)) = $1 ] ; then
    status="yes"
   fi
  fi
done
# End by printing the status.
echo $status
}

# Start at 2.
number=2

while [ 1 ] ; do
# This first if test, test if the number is dividable through 1 and if it's dividable through itself.
  if [ $(($number/1)) = $number -a $(($number/$number)) = 1 ] ; then
   if [ $(throughnoother $number) != "yes" ] ; then
    # If all tests are succesful, then it must be a prime number, print it!
    echo "$number"
   fi
  fi
# Now increase the number to be tested by one.
number=$(($number+1))
done
done

How to burn an ISO file on a DVD in Mac OS X using Disk Utility

Because I alway forget how this exactly works, here is how to get and ISO image burned onto a DVD in Mac OS X. You will be using the application Disk Utility as it is installed by default on Mac OS X.

  1. Download the .ISO file to your computer. Use the desktop to find it back easily, but any other place will work just fine!
  2. Open Disk Utility. You can find this application in Utilities.
  3. Drag the .ISO file onto the left column (white) of Disk Utility. This tells the Disk Utility to do something with that ISO file.
  4. Highlight the image in the Disk Utility.
  5. Click Burn from the toolbar on top. Another screen will pop up, hit Burn to start burning.

Enable an Apple Mac OS X machine as a syslog server

Here is a small howto that describes how your Mac OS X machine can also receive logs from remote devices such as an Apple Airport Extreme. There are some howto's available online, but I guess that somethings have changed in 10.5, none seem to work perfectly.

Change syslogd configuration

# echo "local0.notice /var/log/airport.log" >> /etc/syslog.conf

Touch the logfile

# touch /var/log/airport.log

Change syslogd startup procedure

At the end of the file, uncomment the part to accept remote logging.

# cat /System/Library/LaunchDaemons/com.apple.syslogd.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.apple.syslogd</string>
    <key>OnDemand</key>
    <false/>
    <key>ProgramArguments</key>
    <array>
<!--
Un-comment the following lines to run syslogd with a sandbox profile.
Sandbox profiles restrict processes from performing unauthorized
operations; so it may be necessary to update the profile
(/usr/share/sandbox/syslogd.sb) if any changes are made to the syslog
configuration (/etc/syslog.conf).
-->
<!--
<string>/usr/bin/sandbox-exec</string>
<string>-f</string>
<string>/usr/share/sandbox/syslogd.sb</string>
-->
<string>/usr/sbin/syslogd</string>
    </array>
<key>MachServices</key>
<dict>
<key>com.apple.system.logger</key>
<true/>
</dict>
<key>Sockets</key>
<dict>
<key>AppleSystemLogger</key>
<dict>
<key>SockPathName</key>
<string>/var/run/asl_input</string>
<key>SockPathMode</key>
<integer>438</integer>
</dict>
<key>BSDSystemLogger</key>
<dict>
<key>SockPathName</key>
<string>/var/run/syslog</string>
<key>SockType</key>
<string>dgram</string>
<key>SockPathMode</key>
<integer>438</integer>
</dict>
<!--
Un-comment the following lines to enable the network syslog protocol listener.
-->
<key>NetworkListener</key>
<dict>
<key>SockServiceName</key>
<string>syslog</string>
<key>SockType</key>
<string>dgram</string>
</dict>
</dict>
</dict>
</plist>

Restart syslogd

# launchctl unload /System/Library/LaunchDaemons/com.apple.syslogd.plist
# launchctl load /System/Library/LaunchDaemons/com.apple.syslogd.plist

Open the firewall

Go the the System Preferences, click Security, open the Firewall tab and click the +. Select the file /usr/bin/syslog.
If you are unable to select the /usr directory, try this hack by opening a terminal and typing:

$ cd
$ ln -s /usr/bin
$ ln -s /usr/sbin

Now you can select the file (in your home directory) ./bin/syslog and ./sbin/syslogd

Configure remote devices

Now tell the remote devices (Like the Apple Airport Extremes) to dump their log at the IP address of your Mac OS X machine.

See the result

Now either open the application "Console" or from a terminal, run "tail -f /var/log/airport.log" to see the results as they come in.

Using SSH is not so secure as we expect it to be

It seems there are some negative reports related to the usage of SSH lately:

My faith in SSH as a protocol is huge, reconsidering the usage of SSH might be a good idea, although there are not many alternatives at the moment.

Using rsync from cron with ssh keys that have a passphrase

It took me some time to figure this one out, as everybody is using rsync and ssh-keys without passphrases, but I insist that an ssh-key should have a passphrase.

In my first attemts I got this error messages mailed to me by crontab:

Permission denied (gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive).

Here are the steps to automate a backup initiated from crontab using rsync, SSH and ssh-keys with a passphrase:

  1. Make a set of SSH keys.
  2. Setup SSH to use the agent automatically.
  3. Login once as the user who's cron will run the backup script. You will be asked for a passphrase. When the machine reboots, you will need to login once more, to enter the passphrase again.
  4. Make a backup script that includes some SSH variables.
    This script could be as simple as this:
    . /home/username/.ssh/variables
    rsync -avz --delete /data/ example.com:data

    N.B. This variables file only contains these lines:
    SSH_AUTH_SOCK=/tmp/ssh-DmFcb18036/agent.18036; export SSH_AUTH_SOCK;
    SSH_AGENT_PID=18037; export SSH_AGENT_PID;
    echo Agent pid 18037;
  5. Put that script in crontab.

That should do it for you, as it works like a charm for me!

About Consultancy Articles Contact




References Red Hat Certified Architect By Robert de Bock Robert de Bock
Curriculum Vitae By Fred Clausen +31 6 14 39 58 72
By Nelson Manning [email protected]
Syndicate content