May 2013
M T W T F S S
« Apr    
 12345
6789101112
13141516171819
20212223242526
2728293031  
100

Refs

Categories

Archives

9,331slm
●5 ●38 ●132
 

[one-liner]: Debugging Bash Scripts

Background

From time to time it’s useful if you can turn up the debugging messages that come from Bash, when working out either interactive or shell script problems. Here are 2 methods that can help in getting down to the details.

Solution

There are essentially 2 methods.

Method #1: -x method

When writing a shell script you’ll sometimes want to turn on line by line debugging. There’s basically 2 ways to to this.

Before we get started, suppose we have this sample script, myscript.bash:

1
2
3
4
#!/bin/bash
 
echo "hi"
echo "bye"

First you can run your script like so:

1
2
3
4
5
% bash -x myscript.bash
+ echo hi
hi
+ echo bye
bye

As an alternative you can add the following line, set -x to the top of our shell script to enable debugging as well:

1
2
3
4
5
#!/bin/bash
 
set -x
echo "hi"
echo "bye"
1
2
3
4
5
% ./myscript.bash
+ echo hi
hi
+ echo bye
bye
Method #2: env SHELLOPTS=xtrace …

This approach sets the env. variable SHELLOPTS=xtrace which has the same effect as using bash -x.

For example:

1
2
3
4
5
% env SHELLOPTS=xtrace ./myscript.bash 
+ echo hi
hi
+ echo bye
bye

You can also use this technique to debug your bash environment (think .bashrc and .bash_profile) like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
% env SHELLOPTS=xtrace bash
...
...
+++++ line='complete -f -X
'\''!*.@(zip|[ejw]ar|exe|pk3|wsz|zargo|xpi|sxw|o[tx]t|od[fgpst]|epub|apk)'\''
unzip zipinfo'
+++++ line=' unzip zipinfo'
+++++ list=("${list[@]}" $line)
+++++ read line
+++++ '[' 'complete -f -X '\''*.Z'\'' compress znew' '!=' 'complete -f
-X '\''*.Z'\'' compress znew' ']'
+++++ line='complete -f -X '\''*.Z'\'' compress znew'
+++++ line='complete -f -X '\''*.Z'\'' compress znew'
+++++ line=' compress znew'
+++++ list=("${list[@]}" $line)
+++++ read line
+++++ '[' ' zcmp, zdiff, z*grep, zless, zmore intentionally not here,
see Debian: #455510' '!=' '# zcmp, zdiff, z*grep, zless, zmore
intentionally not here, see Debian: #455510' ']'
...
...

Here you can see every command getting executed from the system and user’s .bashrc and .bash_profile as bash starts up.

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

[one-liner]: How do you capture the status of a command ($?) in Bash, when run through a pipe?

Background

While answering questions on the stackexchage website Unix & Linux I saw the following question which was about something I’d encountered, but until today never knew how to accomplish, so I’m posting it here for my own reference in the future.

The question?

How do you get the exit status ( $? ) from the command haconf -makerw in place of grep? i.e. what need to add in to my syntax in order to understand if haconf -makerw succeeded?

1
2
    haconf -makerw | grep -iq "Cluster already writable"
        # echo $? ( will print the exe status from haconf -makerw  )

Solution

There are 3 ways of doing this. However your current setup should work. The reason here being that the grep won’t match anything if the command fails, so grep will return with status 1 (unless the program always shows that text no matter what).

Pipefail

The first way is to set the pipefail option. This is the simplest and what it does is basically set the exit status $? to the exit code of the last program to exit non-zero (or zero if all exited successfully).

1
2
3
4
5
# false | true; echo $?
0
# set -o pipefail
# false | true; echo $?
1
$PIPESTATUS

Bash also has a variable called $PIPESTATUS which contains the exit status of all the programs in the last command.


…. Continue reading → [one-liner]: How do you capture the status of a command ($?) in Bash, when run through a pipe? »»

[one-liner]: Overview of Bash I/O Redirection

Background

I recently came across this question on the Stackexchange site Unix & Linux. This question was interesting in the sense that it covered much of the I/O redirection facilities that are available in the Bash Shell, so for posterity sake I’m adding my answer to this question here on my blog.

Solution

  • a number 1 = standard out (i.e. STDOUT)
  • a number 2 = standard error (i.e. STDERR)
  • if a number isn’t explicitly given, then number 1 is assumed by the shell (bash)

First let’s tackle the function of these. For reference see the Advanced Bash-Scripting Guide.

Functions
  • 2>&-
- The general form of this one is M>&-, where “M” is a file descriptor number. This will close output for whichever file descriptor is referenced, i.e. “M”.
  • 2>/dev/null
- The general form of this one is M>/dev/null, where “M” is a file descriptor number. This will redirect the file descriptor, “M”, to /dev/null.
  • 2>&1
- The general form of this one is M>&N, where “M” & “N” are file descriptor numbers. It combines the output of file descriptors “M” and “N” into a single stream.
  • |&
- This is just an abbreviation for 2>&1. It was added in Bash 4.
  • &>/dev/null
- This is just an abbreviation for 2>&1 >/dev/null. It too was added in Bash 4. It redirects file descriptor 2 (STDERR) and descriptor 1 (STDOUT) to /dev/null.
  • >/dev/null
- This is just an abbreviation for 1>/dev/null. It redirects file descriptor 1 (STDOUT) to /dev/null.
Portability to non-bash, tcsh, mksh, etc.

I’ve not dealt much with other shells outside of csh and tcsh. My experience with those 2 compared to bash’s redirection operators, is that bash is superior in that regard. See the tcsh man page for more details.

Of the commands asked about in the Unix & Linux question, none are directly supported by csh or tcsh. You’d have to use different syntaxes to construct similar functions.

References

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

[one-liner]: Using scp to Copy Files that Contain Spaces

Background

Have you ever needed to copy either a single file or a directory that contains spaces using scp? Here are a couple of techniques for how to do this.

Solution

Here are several ways to accomplish the same thing. Copying a entire directory with spaces and copying a file with spaces using scp.

1
2
3
4
5
6
7
# Ways to copy entire directories
% scp -r user@myserver.com:'"/path/with/a/Space In It"' .
% scp -r user@myserver.com:"'/path/with/a/Space In It'" .
% scp -r user@myserver.com:'/path/with/a/Space\ In\ It' .
% scp -r user@myserver.com:"/path/with/a/Space\ In\ It" .
% scp -r user@myserver.com:/path/with/a/Space\\\ In\\\ It .
% scp -r user@myserver.com:"\"/path/with/a/Space In It\"" .
1
2
3
4
5
6
7
# Ways to copy a single file
% scp -r user@myserver.com:'"/path/with/a/Space In It/file with spaces.txt"' .
% scp -r user@myserver.com:"'/path/with/a/Space In It/file with spaces.txt'" .
% scp -r user@myserver.com:'/path/with/a/Space\ In\ It/file with spaces.txt' .
% scp -r user@myserver.com:"/path/with/a/Space\ In\ It/file\ with\ spaces.txt" .
% scp -r user@myserver.com:/path/with/a/Space\\\ In\\\ It/file\\\ with\\\ spaces.txt .
% scp user@myserver.com:"\"/path/with/a/Space In It/file with spaces.txt\"" .
links

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

Example Bash Script which Monitors if a Program has Died, and Restarts it

Background

Here’s a quick script that might prove useful if you need to watch if a program is running and restart it if it stops for whatever reason. It is by no means fault tolerant, but could be adapted to be more so if needed.

Solution

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
#!/bin/bash
 
# NAME: check_dropbox.bash
# DESC: watch if dropbox is running
 
check_process() {
  echo "$ts: checking $1"
  [ "$1" = "" ]  && return 0
  [ `pgrep -n $1` ] && return 1 || return 0
}
 
while [ 1 ]; do
  # timestamp
  ts=`date +%T`
 
  echo "$ts: begin checking..."
  check_process "dropbox"
  [ $? -eq 0 ] && echo "$ts: not running, restarting..." && `dropbox start -i > /dev/null`
  sleep 5
done

Running this script, the output would look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# SHELL #2
% dropbox start -i
Starting Dropbox...Done!
 
# SHELL #1
% check_dropbox.bash
22:07:26: begin checking...
22:07:26: checking dropbox
22:07:31: begin checking...
22:07:31: checking dropbox
 
# SHELL #2
% dropbox stop
Dropbox daemon stopped.
 
# SHELL #1
22:07:36: begin checking...
22:07:36: checking dropbox
22:07:36: hot running, restarting...
22:07:42: begin checking...
22:07:42: checking dropbox

Things to consider with this script, in its current form, it’s way to chatty, so getting rid of the echo statements would be the first things to change. 2nd would probably be the frequency, every 5 secs. is too aggressive. Again it’s just meant as an example of how to get something basic like this going. HTH.

[one-liner]: Calculating & Sorting Disk Usage Using du

Background

Occasionally I want to calculate how much disk space is being consumed with a directory. Here are a couple of simple yet powerful one-liners that achieve this.

Solutions

method #1

This first choice is inefficient in the sense that it basically runs du twice. Once to generate a sorted list of the directories & files by size and a 2nd time to get the directories & files in a human readable format. However it works on the widest variety of distros, since it doesn’t rely on any of the newer switches available to du.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
% du -s * | sort -rn | cut -f2- | xargs -d "\n" du -sh
53G	projects
21G	Desktop
7.2G	VirtualBox VMs
3.7G	db
3.3G	SparkleShare
2.2G	Dropbox
272M	apps
47M	incoming
14M	bin
8.6M	parking_lot
5.7M	rpmbuild
76K	task.ref.pdf
68K	vimdir.tgz
method #2

This approach makes use of a new switch -h that was added to the sort command in versions > 0.75. This approach is much faster than the above method, but won’t work on distros like CentOS 5.x.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
% du -hs * | sort -hr
53G	projects
21G	Desktop
7.2G	VirtualBox VMs
3.7G	db
3.3G	SparkleShare
2.2G	Dropbox
272M	apps
47M	incoming
14M	bin
8.6M	parking_lot
5.7M	rpmbuild
76K	task.ref.pdf
68K	vimdir.tgz

Here’s more information about the -h switch:

‘-h’
‘–human-numeric-sort’
‘–sort=human-numeric’
Sort numerically, first by numeric sign (negative, zero, or positive); then by SI suffix (either empty, or ‘k’ or ‘K’, or one of ‘MGTPEZY’, in that order; see Block size); and finally by numeric value. For example, ’1023M’ sorts before ’1G’ because ‘M’ (mega) precedes ‘G’ (giga) as an SI suffix. This option sorts values that are consistently scaled to the nearest suffix, regardless of whether suffixes denote powers of 1000 or 1024, and it therefore sorts the output of any single invocation of the df, du, or ls commands that are invoked with their –human-readable or –si options. The syntax for numbers is the same as for the –numeric-sort option; the SI suffix must immediately follow the number.

References

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

Page 1 of 3123