Red Hat Linux 7 Unleashed

Red Hat Linux 7 Unleashed

By William Ball

First Example—Automating Data Entry

How can the details work out? Look at an example from the day before I started to write this chapter.

Problem and Solution

A client wanted to enhance an online catalog to include thumbnail pictures of the merchandise. After a bit of confusion about what this really meant, I realized that I needed to update a simple database table of products to include a new column (or attribute or value) that would specify the filenames of the thumbnails. The database management system has a couple of interactive front ends, and I'm a swift typist, so it probably would have been quickest to point and click my way through the 200 picture updates. Did I do that? Of course not—what happened later proved the wisdom of this decision. Instead, I wrote a shell script to automate the update, which is shown in Listing 26.1.

Example 26.1. Shell Script That Updates a Database

# picture names seem to look like { $DIR/137-13p.jpg,$DIR/201-942f.jpg,...}
# The corresponding products appear to be { 137-13P, 201-942F, ...}
DIR=/particular/directory/for/my/client

:      # Will we use .gif-s, also, eventually?  I don't know.
 for F in $DIR/*.jpg
 do
        # BASE will have values { 137-13p,201-942f, ...}

   BASE=`basename $F .jpg`
       # The only suffixes I've encountered are 'p'and 'f', so I'll simply
       #     transform those two.
       # Example values for PRODUCT:  { 137-13P, 201-942F, ...}
  PRODUCT=`echo $BASE | tr pf PF`
       # one_command is a shell script that passes a line of SQL to the DBMS.
  one_command update catalog set Picture = "'$DIR/$BASE.jpg'" where Product = "'$PRODUCT'"
 done

As it turned out, the team decided within a couple days that the pictures needed to be in a different directory, so it was only a few seconds' work to update the penultimate line of the script and add a comment such as this and rerun it:

     ...
          # Do *not* include a directory specification in Picture; that will be known
          #     only at the time the data are retrieved.
     one_command update catalog set Picture = "'$BASE.jpg'" where Product = "'$PRODUCT'"
done

It's inevitable that we'll someday have more pictures to add to the database or will want reports on orphaned pictures (those that haven't been connected yet to any product). Then this same script, or a close derivative of it, will come into play again.

Analysis of the Implementation

Now work through the example in Listing 26.1 in detail to practice the automation mentality.

Do you understand how the script in Listing 26.1 works? Chapter 25, "Shell Scripting," explains shell processing and Appendix B, "Top Linux Commands and Utilities," presents everything you're likely to need for the most commonly used UNIX utilities. You can always learn more about these by reading the corresponding man pages or any of the fine books available on shell programming. The most certain way to learn, of course, is to experiment on your own. For example, if you have any question about what man tr means by "translation," it's an easy matter to experiment, such as with this:


   # tr pf PF <<HERE
abcopqOPQ
FfpPab
HERE

You can conclude that you're on the right track when you see the following:

abcoPqOPQ
FFPPab

This is one of the charms of relying on shells for automation; it's easy to bounce between interaction and automation, which shapes a powerful didactic perspective and a check on understanding.

The sample product catalog script in Listing 26.1 is written for sh processing. I strongly recommend this be your target for scripts, rather than ksh, csh, or bash. I prefer any of the latter for interactive command-line use. In automating, however, when I'm often connecting to hosts that don't use Red Hat Linux, availability and esoteric security issues have convinced me to code using constructs that sh—and therefore all the shells—recognize. Default Red Hat Linux installations use a link named /bin/sh that points to /bin/bash. All the work in this chapter is written so that the scripts will function properly no matter what the details are of your host's configuration. Chapter 25 gives more details on the differences among shells.

Did I really include the inline comments, the lines that begin with #, when I first wrote the script in Listing 26.1? Yes. I've made this level of source-code documentation a habit, and it's one I recommend to you. If your life is at all like mine, telephones ring, co-workers chat, and power supplies fail; I find it easier to type this much detail as I'm thinking about it, rather than risk having to re-create my thoughts in case of an interruption. It's also much easier to pick up the work again days or weeks later. Writing for human readability also eases the transition when you pass your work on to others.

Listing 26.1 begins by assigning a shell variable DIR in the third line. It's good practice to make such an assignment, even for a variable (apparently) used only once. It contributes to self-documentation and generally enhances maintainability; it's easy to look at the top of the script and see immediately on what magic words or configuration in the outside environment (/particular/directory/for/my/client in this case; see the third line down) the script depends.

Many of the jobs you'll want to accomplish involve a quantifier: "change all…," "correct every…," and so on. The shell's looping constructs, for and while, are your friends. You'll make almost daily use of them.

basename and tr are universally available and widely used. tr, like many UNIX utilities, expects to read standard input. If you have information in shell variables, you can feed tr the information you want, either through a pipe from echo, as in the example, or an equivalent:


   echo $VARIABLE | tr [a-z] [A-Z]

You can also do it with a so-called HERE document, such as this one:


   tr [a-z] [A-Z] <<HERE
$VARIABLE
HERE

You could perhaps do it by creating a temporary file:


   echo $VARIABLE >$TMPFILE

   tr [a-z] [A-Z] $TMPFILE

one_command is a two-line shell script written earlier in the day to process SQL commands. Why not inline the body of that script here? Although technically feasible, I have a strong preference for small, simple programs that are easy to understand and correspondingly easy to implement correctly. one_command already has been verified to do one small job reliably, so the script lets it do that job. This fits with the UNIX tradition that counsels combining robust toolkit pieces to construct grander works.

In fact, notice that the example in Listing 26.1 shows the shell's nature as a "glue" language. There's a small amount of processing within the shell in manipulating filenames, and then most of the work is handed off to other commands; the shell just glues together results. This is typical and is a correct style you should adopt for your own scripting.

Certainly, it was pleasant when the filenames changed and I realized I could rework one word of the script, rather than retype the 200 entries. As satisfying as this was, the total benefit of automation is still more profound. Even greater than saving my time are the improvements in quality, traceability, and reusability this affords. With the script, I control the data entering the database at a higher level and eliminate whole categories of error: mistyping, accidentally pushing a wrong button in a graphical user interface, and so on. Also, the script in Listing 26.1 records my procedure, in case it's later useful to audit the data. Suppose, for example, that next year it's decided I shouldn't have inserted any of these references to the database's Picture attribute. How many will have to be backed out? Useful answers—at most, the count of $DIR/*.jpg—can be read directly from the script; there's no need to rely on memory or speculation.

Share ThisShare This

Informit Network