Performance analysis on OpenBSD

You are running OpenBSD and encounter some problems with response time of a machine? Read on. This article is a small list of tools you can use to find out where the problem could be and how to solve it.

A performance problem could be defined as: The response that end-users experiences is slower than expected. I have had many discussion about what a performance problem actually is. For example; database administrators will not consider a load of 1.00 a problem on a machine with one CPU. Most system administrators will consider this is a performance problem. I guess it all depends on the nature of the service.

Okay, be your manager is still at your desk, waiting for you to perform magic, so let's get to it.

  1. Run "top". It will display what processes eat the cpu. When top(1) is running, hit "o" to select a new ordering method and type "cpu" to find the must CPU consuming processes. Other variables instead of cpu are "size" and "res" to find out about memory usage, "time" to see what process has been running for so long and "pri" to find out what the priority was assigned to a process. Focus on cpu, size and res.
  2. Run "systat vm". systat(1) will give you information on activity of the disks and interrupts. The interrupts will tell you what driver is causing the most interrupts. This could tell you that your network card is being utilized a lot for example.
  3. Run "systat ifstat". This will show you the network activity.

From experience, this is the list of problems that I commonly encounter:

  1. Incorrectly configured network settings, mostly /etc/resolv.conf
  2. Not enough memory
  3. Disks being slower than expected.

If you can't figure out what is wrong, adding memory never hurts.

When you would like to go a step further, try ktrace(1). Start it with "ktrace -p PID" where PID is the process number of a suspicious process. ktrace(1) dumps all output in a file ktrace.out, which can grow "really" fast. Use "ktrace -C" to stop the tracing.
kdump(1) is used to read a ktrace(1) file. Use "kdump -f ktrace.out" to read and print the file ktrace.out generated with ktrace(1). The output is enormous, so pipe it through less(1) or grep(1) to make it readable.

What I see often is system administrator using ping(8) to analyze network performance. I think ping is a very bad tool for this purpose, better use time(1) and telnet(1) or some script to check the service that clients are complaining about.