Me in IT UNIX/Linux Consultancy is based in Breukelen, The Netherlands and specialized in UNIX and Linux consultancy. Experience with Red Hat Enterprise Linux (Red Hat Certified Architect), Fedora Project, CentOS, OpenBSD and related released Open Source products makes Me in IT UNIX/Linux Consultancy a great partner in implementing, maintaining and upgrading your environment.

Open Source software is an important aspect of any Linux distribution. Me in IT UNIX/Linux Consultancy tries to use Open Source software where possible and tries to share experiences actively. In the articles section you will find many UNIX/Linux adventures shared for others to benefit.

Check your websites PageRank and Backlink quickly

Check out the Xinu website, which is a collection of webtools and checks gathered at one page. Simple and efficient!

So, check your website/blog with xinu.

Logo creator

Me in IT - hosting web 2.0 logo
Nothing shocking, but this site lets you create a logo in web 2.0 style. Check the Web 2.0 - Logo Creatr website.

Sending mail from command line

There are a few tools available to send mail from the command line to an email-address. Here is "mail" for example:

$ cat file
Hello World
$ cat file | mail -s "Here is my email" [email protected]
$

Now this is very nice, but it's not possible to set the from (From:) address using mail. To do that you'd need a more sophisticated solution, using Sendmail.

First you will have to modify that file you'd like to send. This file should include the Headers:

$ cat file
From: [email protected]
To: [email protected]
Subject: Here is my email

Hello World

Now you will have to send this file using sendmail. N.B. Another email address is specified here, this is the actual address where your email will be sent to. This means you can forge everything in the Headers. (In the file)

$ /usr/sbin/sendmail [email protected] < file
$

Well, hope that helps you, it sure helped me when creating reports to send to customers.

Drupal slow, mysql query error

That was a difficult problem; On of my customers website was very slow. I personally created that page, so every piece of code was "known" to me.

My initial thought was that Drupal or some module were slowing down this website. Turns out I made a mistake. I discovered this by reviewing the mysql_slow_query logs. The initial query contained an error:

SELECT files.filepath, node.nid, node.title from files, term_node, term_data,node, node_counter
WHERE files.nid=term_node.nid
AND term_node.tid=term_data.tid
AND node.nid=term_node.nid
AND term_data.tid=65
AND files.filename="thumbnail"
ORDER BY rand() LIMIT 8

Do you see that the node_counter is opened, but not being used anywhere? After removing the bolded text, everything returned to normal. The query was rewritten to this version:

SELECT files.filepath, node.nid, node.title from files, term_node, term_data,node
WHERE files.nid=term_node.nid
AND term_node.tid=term_data.tid
AND node.nid=term_node.nid
AND term_data.tid=65
AND files.filename="thumbnail"
ORDER BY rand() LIMIT 8

Cool webcomics

Likely for years around, but new to me it very nice web comic. Check out this image for example!
A webcomic about the 90s.

Importing txt/cvs into Microsoft Visio

When you are in need of a graphical overview of something and you are able to script it, chances are you are able to import this information into Microsoft Visio.

Here is an example case; You would like to see what what objects link to other objects, not in text, but graphically.

Imagine this is the data:

ONE is connected to TWO
TWO is connected to THREE
THREE is connected to ONE

Read more to create this simple graph:
An example image of a graph.

  1. Convert this information into this (Visio compatible) format:
  2. Shape,"ONE","","this shape is called ONE"
    Shape,"TWO","","this shape is called TWO"
    Shape,"THREE","","this shape is called THREE"
    Link,"","","","ONE","TWO"
    Link,"","","","TWO","THREE"
    Link,"","","","THREE","ONE"
  3. Save the file
  4. Open Visio
  5. Click Open
  6. Set Files of type to Text Files and click Open.
  7. Set the Field Separator to a comma(,) and click OK.
  8. Drag things around until you have a clear overview.
  9. That is it, but you might need to know a bit more. Add this line on top of the text file for automatic placement.

    PlacementStyle,4

    To use a template like Active Directory, use the following syntax. (on top of the text file)

    Template,"Active Directory.vst"

    To indicate an object, like a "site" instead of the default simple box, use the third field in the Shape line:

    Shape,"site3","site","The long version of site3"

    To find out what object names can be used, right-click an object within Visio, and check the properties.

    To add some text on a line, use the fourth field in the Link line:

    Link    ""      ""      "30"    "site2"        "site3"

    (This actually draws a line between site2 and site3, and prints 30 on that line.)

    Good luck making cool layouts!

Cool book: Classic Shell Scripting (O'reilly)

Not trying to be commercial here, but please check out this book about Classic Shell Scripting. It's published by O'reilly, which has produced quite some cool books.

This book focuses on the shell (bash, bourn shell) scripting as you might have guessed, judged by the title. Why this book is better, compare these two pieces of code:

#!/bin/bash

# ugly code example

# Count the number of lines in `w`.
who_raw=`w | wc -l`

# Two lines are used for headers, subtract them.
who=`echo $who_raw-2 | bc`

echo "There are $who users logged in."

#!/bin/bash

# better code example

echo "There are $(($(w | wc -l)-2)) users logged in."

And, trust me, after reading (parts of) this book, the "better code" could be approved on many points.

The result of reading this book is faster, cleaner, more "native" shell scripts.

Replacing things in MySQL

UPDATE The proper trick is described here:

update table set field = replace(field, 'original string','new string');

Here is how to replace some text in mysql, using some shell commands.
I am sure somebody can tell me how this can be done in mysql, not using these shell commands.

This specific query is created for Drupal, but it can be modified for any database model.

mysql -u USER -p DATABASE -B -e "select nid,title from node;" \
| sed 's/\.jpg//;s/\.JPG//'| while read nid title ; do echo "update node \
set title=\"$title\" where nid=$nid;"; done > file.txt

The file.txt will look like this:

update node set title="DSCF0233" where nid=1767;
update node set title="DSCF0236" where nid=1768;
update node set title="DSCF0256" where nid=1769;

To import this back into mysql:

mysql -u USER -p DATABASE &lt; file.txt

Traffic Jam Graphs

Check out this graph, it represents the all traffic jams in kilometers in the Netherlands. I use it to determine the best time to leave home for office and vice versa.

graph of current traffic jams


There are a few steps in creating these graphs, let's go over them one by one.

Step 1: Get the data

First thing ofcouse is to get (and store) the data. Think of a simple script that output some information, like this:

#!/bin/sh
date=$(date +'%s')
value=$(uptime | awk '{print $NF}')
echo "$date $value" >> alldata.txt

Step 2: Store the data frequently

Now, that was easy, next thing is to add your script to the crontab, so it's run every 5 minutes or so. You can edit the crontab like this:

crontab -e

You will now get an editor on your screen, add this line to it:
*/5 * * * * /home/you/script.sh

Change the path name to reflect your situation, same for the scriptname.
You will notice that this file alldata.txt will be filled with lines. Good, you'll need it.

Step 3: Create an RRD database

Here is where the magic begins. Please read the RRD manuals to get detailed information.

rrdtool create load.rrd DS:load:GAUGE:300:0:U RRA:AVERAGE:0.5:1:10800

What this mean:

  1. rrdtool - The tool that is used to do things with rrd. Download it from the official RRD website.
  2. create - Make a new file used as a round robin database.
  3. load.rrd - Call that file load.rrd. Choose something normal.
  4. DS: - This is a Data Store, you can create more than one store in a file.
  5. load: - Call this Data Store "load".
  6. GAUGE: - The values act like a gauge, they represent a current value. Comparable to a speed meter.
  7. 300 - This interval to store a value in this Data Store is 300 seconds. (5 minutes) This can be modified, and is heavily dependant on what information you want to caputure.
  8. 0: - The minimum value is 0.
  9. U - The maximum value is Unknown. (Making the graph scalable.)
  10. RRA - This is a Round Robin Archive.
  11. 0.5: - Ratio between known and unknown Primary Data Points. Forget it, put it on 0.5.
  12. 1: - Average the value with this number. (1: no average, just the one value.)
  13. 10800 - Store this amount of samples (rows) in the Round Robin Archive. More results in larger files. (But fixed in size, wether it's full or empty.)

You are prepared now to start entering data into this rrd file. Please be aware that you can't enter "old" information into this rrd. Old mean data before this rrd was created. If you have "old" data, you need to create your database with the --start option.

Step 4: Entering data into your rrd

Now, this is actually quite an ease step. Remember you have created that script to store all the data? Lets dump that data into an rrd. (You can also do this without storing it in a file first.)

cat alldata.txt | while read date value ; do
rrdtool update load.rrd "$date":"$value"
done

Step 5: Creating a graph

Finaly some magic, creating a graph with rrdtool.

rrdtool graph load.png --vertical-label load DEF:load=load.rrd:load:AVERAGE LINE2:load#FF0000:'15 minutes load'

What do all these things mean?

  1. rrdtool - Again the program rrdtool.
  2. graph - Now make a graph.
  3. load.png - Call the file load.png
  4. --vertical-label load - Put "load" on the left side, vertically.
  5. DEF: - Define a variable
  6. load=load.rrd: - The variable is called "load", it comes from the load.rrd file.
  7. AVERAGE - The line represents an average of the measured values.
  8. LINE2: - Draw a 2-pixel line. (LINE1, AREA are other cool things to play with.)
  9. load#FF0000: This line represents the load variable, and is colored #FF0000.
  10. '15 minutes load' - In the legend, call this color: "15 minutes load".

This step determines the looks of your graph, including the scale. (hour/day/week/etc.) To set the scale, you will need the --start and --end options. Without these options, a week will be plotted.

To get the cool looks (a red line, filled with an orange color, combine the LINE2 and AREA items, like this:

rrdtool graph load.png --vertical-label load DEF:load=load.rrd:load:AVERAGE AREA:load#FF6600: LINE2:load#FF0000:'15 minutes load'

You are done now, getting this all together, you'll want to script some things. Use the crontab facility to initiate these scripts.

About Consultancy Articles Contact




References Red Hat Certified Architect By Robert de Bock Robert de Bock
Curriculum Vitae By Fred Clausen +31 6 14 39 58 72
By Nelson Manning [email protected]
Syndicate content