| Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting | ||
|---|---|---|
| Prev | Appendix C. A Sed and Awk Micro-Primer | Next |
Awk is a full-featured text processing language with a syntax reminiscent of C. While it possesses an extensive set of operators and capabilities, we will cover only a couple of these here - the ones most useful for shell scripting.
Awk breaks each line of input passed to it into fields. By default, a field is a string of consecutive characters separated by whitespace, though there are options for changing the delimiter. Awk parses and operates on each separate field. This makes awk ideal for handling structured text files -- especially tables -- data organized into consistent chunks, such as rows and columns.
Strong quoting (single quotes) and curly brackets enclose segments of awk code within a shell script.
1 echo one two | awk '{print $1}'
2 # one
3
4 echo one two | awk '{print $2}'
5 # two
6
7
8 awk '{print $3}' $filename
9 # Prints field #3 of file $filename to stdout.
10
11 awk '{print $1 $5 $6}' $filename
12 # Prints fields #1, #5, and #6 of file $filename. |
We have just seen the awk print command in action. The only other feature of awk we need to deal with here is variables. Awk handles variables similarly to shell scripts, though a bit more flexibly.
1 { total += ${column_number} } |
1 END { print total } |
Corresponding to the END, there is a BEGIN, for a code block to be performed before awk starts processing its input.
The following example illustrates how awk can add text-parsing tools to a shell script.
Example C-1. Counting Letter Occurrences
1 #! /bin/sh
2 # letter-count2.sh: Counting letter occurrences in a text file.
3 #
4 # Script by nyal [nyal@voila.fr].
5 # Used with permission.
6 # Recommented by document author.
7 # Version 1.1: Modified to work with gawk 3.1.3.
8 # (Will still work with earlier versions.)
9
10
11 INIT_TAB_AWK=""
12 # Parameter to initialize awk script.
13 count_case=0
14 FILE_PARSE=$1
15
16 E_PARAMERR=65
17
18 usage()
19 {
20 echo "Usage: letter-count.sh file letters" 2>&1
21 # For example: ./letter-count2.sh filename.txt a b c
22 exit $E_PARAMERR # Not enough arguments passed to script.
23 }
24
25 if [ ! -f "$1" ] ; then
26 echo "$1: No such file." 2>&1
27 usage # Print usage message and exit.
28 fi
29
30 if [ -z "$2" ] ; then
31 echo "$2: No letters specified." 2>&1
32 usage
33 fi
34
35 shift # Letters specified.
36 for letter in `echo $@` # For each one . . .
37 do
38 INIT_TAB_AWK="$INIT_TAB_AWK tab_search[${count_case}] = \"$letter\"; final_tab[${count_case}] = 0; "
39 # Pass as parameter to awk script below.
40 count_case=`expr $count_case + 1`
41 done
42
43 # DEBUG:
44 # echo $INIT_TAB_AWK;
45
46 cat $FILE_PARSE |
47 # Pipe the target file to the following awk script.
48
49 # ----------------------------------------------------------------------------------
50 # Earlier version of script used:
51 # awk -v tab_search=0 -v final_tab=0 -v tab=0 -v nb_letter=0 -v chara=0 -v chara2=0 \
52
53 awk \
54 "BEGIN { $INIT_TAB_AWK } \
55 { split(\$0, tab, \"\"); \
56 for (chara in tab) \
57 { for (chara2 in tab_search) \
58 { if (tab_search[chara2] == tab[chara]) { final_tab[chara2]++ } } } } \
59 END { for (chara in final_tab) \
60 { print tab_search[chara] \" => \" final_tab[chara] } }"
61 # ----------------------------------------------------------------------------------
62 # Nothing all that complicated, just . . .
63 #+ for-loops, if-tests, and a couple of specialized functions.
64
65 exit $?
66
67 # Compare this script to letter-count.sh. |
For simpler examples of awk within shell scripts, see:
That's all the awk we'll cover here, folks, but there's lots more to learn. See the appropriate references in the Bibliography.