July 2014
M T W T F S S
« Mar    
 123456
78910111213
14151617181920
21222324252627
28293031  
153

Refs

Categories

Archives

profile for slm on Stack Exchange, a network of free, community-driven Q&A sites

[one-liner]: Why is Conky reporting a lower CPU frequency, when my CPU frequency is actually much higher?

Background

If you’ve every dealt with Conky you may have gotten a little confused when you’re trying to get it to display your CPU frequency like so:

1
${freq_g cpu0} Ghz

… and Conky is reporting your CPU frequency as 1.12GHz when in fact it’s actually much higher than that, say 2.67GHz. Most likely this is being caused by the CPU governing features that are present in most modern hardware. Here’s a behind the scenes 5 second tour of seeing what’s going on within your Linux Kernel.

Solution

First, from the conky man page.

cpu (cpuN)

CPU usage in percents. For SMP machines, the CPU number can
be provided as an argument. ${cpu cpu0} is the total usage, and ${cpu
cpuX} (X >= 1) are individual CPUs.

freq_g (n)

Returns CPU #n’s frequency in GHz. CPUs are counted from 1.
If omitted, the parameter defaults to 1.

You most likely have something like SpeedStep enabled which is acting like a governor on a car, regulating the speed of the cores inside your CPU. You can confirm that this is going on by looking at the output of this command:

1
2
3
4
5
6
7
8
9
% less /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 37
model name      : Intel(R) Core(TM) i5 CPU       M 560  @ 2.67GHz
stepping        : 5
cpu MHz         : 1199.000
...

The 2 numbers that matter are the 2.67GHz, that the GHz that my CPU is rated to operate at followed by the number 1199.00, this is what my CPU is allowed to run at by the governor setup on my Linux laptop.

You can see what governor is currently configured like so:


…. Continue reading → [one-liner]: Why is Conky reporting a lower CPU frequency, when my CPU frequency is actually much higher? »»

[one-liner]: Checking out a HDD's Health using the Command Line Tool udisk

Background

Here’s a quick tip for checking out the overall health of your computer’s hard drive. It makes use of a little known tool called udisks which provides access to information about storage devices from the D-Bus interface.

Solution

Before we jump into udisks here are some resources that might prove useful when dealing with the design/architecture of udisks.

To see which drives are currently under udisks watchful eye, you can use the following command:

1
2
3
4
5
6
7
8
9
10
11
12
% sudo udisks --enumerate
/org/freedesktop/UDisks/devices/dm_2d0
/org/freedesktop/UDisks/devices/dm_2d1
/org/freedesktop/UDisks/devices/dm_2d2
/org/freedesktop/UDisks/devices/sda1
/org/freedesktop/UDisks/devices/sda2
/org/freedesktop/UDisks/devices/sda3
/org/freedesktop/UDisks/devices/sda4
/org/freedesktop/UDisks/devices/sda5
/org/freedesktop/UDisks/devices/sda6
/org/freedesktop/UDisks/devices/sr0
/org/freedesktop/UDisks/devices/sda

This output shows that I have several partitions that are currently bing monitored, the last bit in the path tells you which device, i.e. sda1, sda2, sda3, etc. These are partitions /dev/sda1, /dev/sda2, etc.

Now on to the actual health output. This next command will show you the overall health of device /dev/sda, i.e. my primary HDD on my laptop:


…. Continue reading → [one-liner]: Checking out a HDD’s Health using the Command Line Tool udisk »»

[one-liner]: How can I Trigger a Notification When a Command Line Job/Process Ends?

Background

Occasionally I need to notify myself when a long running command has finished. Here are a couple of tricks that I use to give me either a audible or visual notification.

Solution

The trick with this is to construct your command like so:

1
% command; command-after &

Then in the command-after position you can do things like this:

1
2
3
4
5
6
7
8
# ring the terminal bell
% tput bel
 
# use espeak to say stuff
% espeak "job's done!"
 
# notify-send (show's a message in upper right)
% notify-send "job's done!"
notify-send ss

notify-send ss

1
2
# zenity (show's a dialog box with OK button)
% zenity --info --text="Job's done"
zenity ss

zenity ss

Additionally if you want to only display a message if the command succeeds or fails you can use these patterns:

1
2
3
4
5
# command succeeded
% command && command-after &
 
# command failed
% command || command-after &

References

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

[one-liner]: Monitoring Network Traffic with Netstat on Linux

Background

This is a quick guide to some not so obvious command-line options available in the Linux netstat(8) command. I hope you enjoy it.

Solution

1. Plain old netstat

Without any command-line arguments, “netstat” shows a list of network connections in your system, including TCP, UDP and UNIX socket connections. If you want to speed up things a bit, use “-n” (numeric) to prevent network lookups and display IP and port numbers instead of names.

1
2
3
4
5
6
7
8
9
10
11
12
% netstat -n
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 192.168.1.20:41157          173.194.73.104:443          TIME_WAIT   
tcp        0      0 192.168.1.20:57224          67.253.170.83:5222          ESTABLISHED 
...
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ]         DGRAM                    7590   @/org/kernel/udev/udevd
unix  2      [ ]         DGRAM                    11188  @/org/freedesktop/hal/udev_event
unix  22     [ ]         DGRAM                    10175  /dev/log
...

2. Seeing all connections with “netstat -a”

Just like the above, but shows all connections, including those in the LISTEN state. Good when you want to see all connections in one shot.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
% netstat -an
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:631                 0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:17500               0.0.0.0:*                   LISTEN      
tcp        0      0 127.0.0.1:2143              0.0.0.0:*                   LISTEN      
tcp        0      0 127.0.0.1:1986              0.0.0.0:*                   LISTEN      
tcp        0      0 127.0.0.1:2025              0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:59081               0.0.0.0:*                   LISTEN      
tcp        0      0 192.168.1.20:57224          67.253.170.83:5222          ESTABLISHED 
...
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ACC ]     STREAM     LISTENING     101544 /home/saml/.dropbox/command_socket
unix  2      [ ACC ]     STREAM     LISTENING     101549 /home/saml/.dropbox/iface_socket
unix  2      [ ACC ]     STREAM     LISTENING     11051  /var/run/cups/cups.sock
...

3. What I am listening to? “netstat -l”

Many people type “netstat -a | grep -i LISTEN“, but “netstat -l” will do the same: filter the output to show sockets in the LISTEN state only. Very useful to quickly see what is being “served” in your box. You can combine this with “-u” to only show UDP connections or “-t” to restrict the output to TCP connections only.

1
2
3
4
5
6
7
8
9
10
11
12
13
% netstat -ln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:631                 0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:17500               0.0.0.0:*                   LISTEN      
...
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ACC ]     STREAM     LISTENING     101544 /home/saml/.dropbox/command_socket
unix  2      [ ACC ]     STREAM     LISTENING     101549 /home/saml/.dropbox/iface_socket
...

4. Who’s using that socket? “netstat -p”

With “-p”, netstat shows what program/pid is using a given socket. Very handy to find out who’s listening on a port or holding a connection open. A personal favorite of mine is “netstat -lput”, which displays all TCP and UDP sockets in the LISTEN state, plus the name and pid of the program listening on that socket.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
% netstat -lnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:631                 0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:17500               0.0.0.0:*                   LISTEN      4185/dropbox        
tcp        0      0 127.0.0.1:2143              0.0.0.0:*                   LISTEN      31544/ssh           
...
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node PID/Program name    Path
unix  2      [ ACC ]     STREAM     LISTENING     101544 4185/dropbox        /home/saml/.dropbox/command_socket
unix  2      [ ACC ]     STREAM     LISTENING     101549 4185/dropbox        /home/saml/.dropbox/iface_socket
unix  2      [ ACC ]     STREAM     LISTENING     11051  -                   /var/run/cups/cups.sock
...
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
% netstat -lput
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 *:sunrpc                    *:*                         LISTEN      1403/rpcbind        
tcp        0      0 *:ssh                       *:*                         LISTEN      1753/sshd           
tcp        0      0 *:ipp                       *:*                         LISTEN      1578/cupsd          
tcp        0      0 *:db-lsp                    *:*                         LISTEN      4185/dropbox        
tcp        0      0 localhost.localdomain:lv-jc *:*                         LISTEN      31544/ssh           
tcp        0      0 localhost.loc:licensedaemon *:*                         LISTEN      2764/mono           
tcp        0      0 localhost.localdoma:ellpack *:*                         LISTEN      31544/ssh           
tcp        0      0 *:59081                     *:*                         LISTEN      1508/rpc.statd      
tcp        0      0 *:sunrpc                    *:*                         LISTEN      1403/rpcbind        
tcp        0      0 *:ssh                       *:*                         LISTEN      1753/sshd           
tcp        0      0 *:ipp                       *:*                         LISTEN      1578/cupsd          
tcp        0      0 *:46136                     *:*                         LISTEN      1508/rpc.statd      
tcp        0      0 localhost.localdomain:lv-jc *:*                         LISTEN      31544/ssh           
tcp        0      0 localhost.localdoma:ellpack *:*                         LISTEN      31544/ssh           
udp        0      0 *:bootpc                    *:*                                     30569/dhclient      
udp        0      0 *:sunrpc                    *:*                                     1403/rpcbind        
udp        0      0 greeneggs.bubba.net:ntp     *:*                                     30725/ntpd          
udp        0      0 localhost.localdomain:ntp   *:*                                     30725/ntpd          
udp        0      0 *:ntp                       *:*                                     30725/ntpd          
udp        0      0 *:32944                     *:*                                     1480/avahi-daemon:  
udp        0      0 *:ipp                       *:*                                     1578/cupsd          
udp        0      0 *:727                       *:*                                     1403/rpcbind        
udp        0      0 *:836                       *:*                                     1508/rpc.statd      
udp        0      0 *:db-lsp-disc               *:*                                     4185/dropbox        
udp        0      0 *:mdns                      *:*                                     1480/avahi-daemon:  
udp        0      0 *:30320                     *:*                                     30569/dhclient      
udp        0      0 *:60010                     *:*                                     1508/rpc.statd      
udp        0      0 *:sunrpc                    *:*                                     1403/rpcbind        
udp        0      0 localhost.localdomain:ntp   *:*                                     30725/ntpd          
udp        0      0 fe80::226:c7ff:fe85:ntp     *:*                                     30725/ntpd          
udp        0      0 *:ntp                       *:*                                     30725/ntpd          
udp        0      0 *:49602                     *:*                                     1508/rpc.statd      
udp        0      0 *:727                       *:*                                     1403/rpcbind        
udp        0      0 *:6095                      *:*                                     30569/dhclient

5. Details, lots of details: “netstat -e”

If you really want to see what’s going on, add the “-e” command-line switch to your netstat command. It will cause “extra” information to be printed, such as the username, process name and pid, and the inode number of each of your sockets.


…. Continue reading → [one-liner]: Monitoring Network Traffic with Netstat on Linux »»

Example Bash Script which Monitors if a Program has Died, and Restarts it

Background

Here’s a quick script that might prove useful if you need to watch if a program is running and restart it if it stops for whatever reason. It is by no means fault tolerant, but could be adapted to be more so if needed.

Solution

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
#!/bin/bash
 
# NAME: check_dropbox.bash
# DESC: watch if dropbox is running
 
check_process() {
  echo "$ts: checking $1"
  [ "$1" = "" ]  && return 0
  [ `pgrep -n $1` ] && return 1 || return 0
}
 
while [ 1 ]; do
  # timestamp
  ts=`date +%T`
 
  echo "$ts: begin checking..."
  check_process "dropbox"
  [ $? -eq 0 ] && echo "$ts: not running, restarting..." && `dropbox start -i > /dev/null`
  sleep 5
done

Running this script, the output would look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# SHELL #2
% dropbox start -i
Starting Dropbox...Done!
 
# SHELL #1
% check_dropbox.bash
22:07:26: begin checking...
22:07:26: checking dropbox
22:07:31: begin checking...
22:07:31: checking dropbox
 
# SHELL #2
% dropbox stop
Dropbox daemon stopped.
 
# SHELL #1
22:07:36: begin checking...
22:07:36: checking dropbox
22:07:36: hot running, restarting...
22:07:42: begin checking...
22:07:42: checking dropbox

Things to consider with this script, in its current form, it’s way to chatty, so getting rid of the echo statements would be the first things to change. 2nd would probably be the frequency, every 5 secs. is too aggressive. Again it’s just meant as an example of how to get something basic like this going. HTH.

Improving the Wordpress Plugin Statpress Reloaded's Performance

Since setting up this blog some 2+ years ago I’ve been using the WordPress plugin, Statpress Reloaded, to keep tabs on how much traffic it receives. It’s purely just for vanity sake. Now for anyone that’s used Statpress Reloaded, you’ll know that it’s a nice plugin for what it shows, mainly real-time stats, but it’s a dog when it comes to performance.

My instance had been getting slower and slower as Statpress Reloaded’s database table (wordpress.wp1_statpress) had been getting bigger and bigger until one day it just stopped working. Going into the MySQL console I could see new data was getting added but the table (250MB+) had just gotten too big.

This was a low priority task until tonight, I figured I’d hunker down and see if I could get it working again. Turns out fixing it so that it’s at least usable was pretty easy.

First I connected to my wordpress blog MySQL DB:

1
2
3
4
5
6
7
8
9
10
11
12
% mysql -u root -p wordpress
Enter password: 
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2038049
Server version: 5.0.77 Source distribution
 
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
 
mysql>

Initially this was what my statpress table looked like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
mysql> desc wp1_statpress;
+--------------+--------------+------+-----+---------+----------------+
| Field        | Type         | Null | Key | Default | Extra          |
+--------------+--------------+------+-----+---------+----------------+
| id           | mediumint(9) | NO   | PRI | NULL    | auto_increment | 
| date         | tinytext     | YES  |     | NULL    |                | 
| time         | tinytext     | YES  |     | NULL    |                | 
| ip           | tinytext     | YES  |     | NULL    |                | 
| urlrequested | text         | YES  |     | NULL    |                | 
| agent        | text         | YES  |     | NULL    |                | 
| referrer     | text         | YES  |     | NULL    |                | 
| search       | text         | YES  |     | NULL    |                | 
| nation       | tinytext     | YES  |     | NULL    |                | 
| os           | tinytext     | YES  |     | NULL    |                | 
| browser      | tinytext     | YES  |     | NULL    |                | 
| searchengine | tinytext     | YES  |     | NULL    |                | 
| spider       | tinytext     | YES  |     | NULL    |                | 
| feed         | tinytext     | YES  |     | NULL    |                | 
| user         | tinytext     | YES  |     | NULL    |                | 
| timestamp    | tinytext     | YES  |     | NULL    |                | 
| threat_score | smallint(6)  | YES  |     | NULL    |                | 
| threat_type  | smallint(6)  | YES  |     | NULL    |                | 
+--------------+--------------+------+-----+---------+----------------+

I ran the following command which changes the column data types from tinytext (predominately) to much more specific data types (varchars). Switching the data types cuts down on the amount of work MySQL has to do when working with the different columns.

1
2
3
4
5
6
7
8
9
10
11
alter table wp1_statpress modify column date int(8) NULL NULL,
modify column time time NULL NULL,  
modify column ip varchar(15) NULL NULL,
modify column nation varchar(10) NULL NULL,
modify column os varchar(64) NULL NULL,
modify column browser varchar(64) NULL NULL,
modify column searchengine varchar(64) NULL NULL,
modify column spider varchar(64) NULL NULL,
modify column feed varchar(32) NULL NULL,
modify column user varchar(64) NULL NULL, 
modify column timestamp timestamp NULL NULL;

Additionally I added the following DB indexes, again to improve the overall performance of this table within MySQL:

1
2
3
4
5
6
7
8
9
10
alter table wp1_statpress add index spider_nation (spider, nation),
add index ip_date (ip, date), 
add index (agent(255)), 
add index (search(255)),
add index (referrer(255)),
add index feed_spider_os (feed, spider, os),                  
add index (os),
add index date_feed_spider (date, feed, spider),
add index feed_spider_browser (feed, spider, browser),
add index (browser);

Conclusions

Now my statpress table looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
mysql> desc wp1_statpress;
+--------------+--------------+------+-----+---------+----------------+
| Field        | Type         | Null | Key | Default | Extra          |
+--------------+--------------+------+-----+---------+----------------+
| id           | mediumint(9) | NO   | PRI | NULL    | auto_increment | 
| date         | int(8)       | YES  | MUL | NULL    |                | 
| time         | time         | YES  |     | NULL    |                | 
| ip           | varchar(15)  | YES  | MUL | NULL    |                | 
| urlrequested | text         | YES  |     | NULL    |                | 
| agent        | text         | YES  | MUL | NULL    |                | 
| referrer     | text         | YES  | MUL | NULL    |                | 
| search       | text         | YES  | MUL | NULL    |                | 
| nation       | varchar(10)  | YES  |     | NULL    |                | 
| os           | varchar(64)  | YES  | MUL | NULL    |                | 
| browser      | varchar(64)  | YES  | MUL | NULL    |                | 
| searchengine | varchar(64)  | YES  |     | NULL    |                | 
| spider       | varchar(64)  | YES  | MUL | NULL    |                | 
| feed         | varchar(32)  | YES  | MUL | NULL    |                | 
| user         | varchar(64)  | YES  |     | NULL    |                | 
| timestamp    | timestamp    | YES  |     | NULL    |                | 
| threat_score | smallint(6)  | YES  |     | NULL    |                | 
| threat_type  | smallint(6)  | YES  |     | NULL    |                | 
+--------------+--------------+------+-----+---------+----------------+

And I’m once again able to access my Statpress Reloaded charts. Previously the charts would take minutes to load, now they come up in under 60 seconds. Down the road I’ll probably have to address this problem again, I still have the ability to delete rows from my statpress table (2+ years of data), It’s currently > 1M rows…yes that’s 1 million!

1
2
StatPress table size: 230,13 MB (1033389 records)
StatPress current time: 2011-07-06 04:29:04

Additionally, there are other plugins that I may investigate switching to down the road:

but in the meantime Statpress Reloaded is working for me in an acceptable way.

References

links
local copies
Page 1 of 212