UNIX AWK and SED Programmer's Interactive Workbook

Description

• Copyright 1999
• Dimensions: 7" x 9-1/4"
• Pages: 500
• Edition: 1st

• Book
• ISBN-10: 0-13-082675-8
• ISBN-13: 978-0-13-082675-6

8267E-0

Start with a hands-on introduction to regular expressions and grep-including egrep and fast grep. Learn key sed fundamentals: addressing, scripting, plus all the commands you'll need for substitution, deletion, printing, appending, listing, reading/writing, and more. Walk step-by-step through awk, writing simple programs, understanding data types, statements, expressions, patterns, actions, functions, control flow, arrays, and I/O-all you need to solve real-world problems!

Master awk and sed programming now!

Totally integrated with a FREE, state-of-the-art UNIX learning Web site!

Every UNIX Interactive Workbook is fully integrated with its own exclusive Web site, giving you all this and more:

• "Test Your Thinking" project solutions and detailed explanations!
• Author's Corner: Your personal connection to this book's expert author!
• Practice questions with instant feedback and explanations!
• An exclusive message board where you can interact with other students!

Just the facts! No endless, boring discussions here! You'll learn hands-on, through practical exercises, self-review questions and real-world answers. Exclusive "Test Your Thinking" projects guarantee you'll go beyond rote knowledge to really master the subject! It's an integrated learning system that's proven to work!

Dozens of exercises cover the real world tasks that matter most!

100s of self-review questions and answers make sure you understand!

0-13-082675-8



Sample Content

Downloadable Sample Chapter

Click here for a sample chapter for this book: 0130826758.pdf

Table of Contents

1. Regular Expressions and Metacharacters.

L A B 1.1 Using the period and backslash metacharacters. Test Your Thinking.

2. Character Class, Hyphen, and Caret Metacharacters.

L A B 2.1 The Character Class, Hyphen, and Caret Metacharacters. Test Your Thinking.

3. What Is Grep and What Does It Do?

L A B 3.1 Getting Started with Grep. Test Your Thinking.

4. Using Grep with Regular Expressions.

L A B 4.1 Regular Expressions and Grep. L A B 4.2 fgrep and egrep. L A B. Using the Anchor Metacharacter. Test Your Thinking.

5. Introduction to sed.

L A B 5.1 Getting Started with sed. Test Your Thinking.

6. Sed Syntax and Addressing.

L A B 6.1 Sed Syntax and Addressing. Test Your Thinking.

7. Sed Commands.

L A B 7.1 The Substitution Command. L A B 7.2 The Delete Command. L A B 7.3 The Append, Insert, and Change Commands. L A B 7.4 The List and Transform Commands. L A B 7.5 The Print and Next Commands. L A B 7.6 The Read and Write Commands. Test Your Thinking.

8. Multiline Pattern Space Commands.

L A B 8.1 The Multiline Next, Delete, and Print Commands. Test Your Thinking.

9. An Awk Program.

L A B 9.1 An Example Awk Program. L A B 9.2 The Structure of an Awk Program. L A B 9.3 Invoking an Awk Program. Test Your Thinking.

10. Basic Awk Language Elements.

L A B 10.1 Awk Statements and Expressions. L A B 10.2 Awk Strings. L A B 10.3 Awk Numbers. L A B 10.4 Coercion. L A B 10.5 awk Assignment operator. Test Your Thinking.

11. Variables.

L A B 11.1 Field Variables. L A B 11.2 User-Defined Variables. L A B 11.3 Built-In Variables. Test Your Thinking.

12. Operators.

L A B 12.1 Arithmetic Operators. L A B 12.2 the Assignment Statement. L A B 12.3 Logical and Relational Operators, and Conditional Expressions. Test Your Thinking.

13. Built-in Functions, Precedence, Associativity, and Machine Limits.

L A B 13.1 Built-In Arithmetic Functions. L A B 13.2 Built-In String Functions. L A B 13.3 Patterns, Precedence, Associativity, and Machine Limits. Test Your Thinking.

14. Awk Control Flow.

L A B 14.1 The If-Else Statement. L A B 14.2 The While and Do Statements. L A B 14.3 The For Statement. L A B 14.4 Loop and Program Control. Test Your Thinking.

15. Awk Arrays and Functions.

L A B 15.1 Awk Arrays. L A B 15.2 Awk User-Defined Functions. Test Your Thinking.

16. Advanced Input and Output.

L A B 16.1 The printf and sprintf Statements. L A B 16.2 Output into Pipes, Files, and the Getline Function. Test Your Thinking.

APPENDIX A Answers to Self-Review Questions.


APPENDIX B Awk, Sed, and Grep Reference.


INDEX.

Preface

Introduction

This book is about three UNIX utilities: grep, sed, and awk. These three utilities enable you to write various terse applications. These UNIX utilities have been around almost as long as the UNIX operating system and yet still are used to solve a variety of tasks even to this day. Grep, awk, and sed can be very useful for file processing; working at the command line, searching within files; working in combination with other UNIX utilities to perform common tasks at the command line; or writing short scripts to solve an application. These are just a few of the tasks for which these utilities can be used. The goal of this book is to introduce and reinforce your expertise in grep, awk, and sed so that you may solve applications that you currently need to implement. In addition, I hope that, by reading this book, when you do have an application to implement, you have enough understanding of the three utilities to consider using them both to solve your application, and to implement a solution. To achieve this goal, the elements of each utility are taught. Each utility's concepts and elements are presented, with detail about the particular syntax, behavior, rules, and nuances of each concept and element. Exercises and exercise discussions then highlight and reinforce the elements and concepts.

All three utilities share common properties. They all can work on standard input, standard output, or user-supplied input files. They may interact with the UNIX environment by utilizing pipes. And all work with regular expressions.

How This Book Is Organized

Regular expressions are covered first in this book, because all three utilities require a knowledge of them. If you are not familiar with regular expressions, these chapters will introduce them and go over the various metacharacters that are used with them. Although this coverage is extensive, it is not intended to provide the reader with advanced knowledge on the subject. It is meant for beginners or intermediate users of regular expressions. Whenever covering grep, sed, and awk and using regular expressions, I assume that you have not encountered regular expressions before. If you already have encountered regular expressions, then you may skim over the material or skip it altogether.

The next section describes grep. Grep is a utility that is best suited for searching files; thus, I first describe grep with the use of regular expressions. Next, I cover using grep in conjunction with pipes and other UNIX programs. Finally, I discuss the various options with which you may invoke grep.

The next part of the book discusses sed. Sed addressing is covered, as well as the various sed commands that are available within sed. Sed commands provide more functionality than is available in grep and can be used for a variety of applications. Finally, we will discuss more advanced sed commands, such as the multiline pattern space.

Awk is covered last. Awk, as opposed to grep and sed, can be considered more closely tied to a general-purpose language rather than a utility or specialized language. The reason is that awk exhibits more constructs and features that general-purpose languages exhibit. These include:

• The ability to control flow to any part of an awk program
• The ability to store values in user-defined variables that reference general storage locations
• The ability to perform arithmetic operations
• The ability to write functions
• The ability to perform output in a user-specified format

Sed and grep either restrict these features or do not include them. Generally, most books take the approach of teaching awk as a utility and teach solely by giving an example utility using an awk feature. The approach taken in this book is to teach awk as a language rather than a utility. Therefore, we will talk about awk data types, variables, built-in functions, arrays, control statements, input and output, and functions. Many pitfalls occur with teaching awk as a utility and providing examples that use awk to solve that particular utility. First, if all you ever need is to solve those particular utilities, then you are set. However, if you need to implement utilities that are different from those provided, you are left figuring out and understanding the language through the examples on your own. A better approach is to teach awk as a language and give examples that reinforce each particular feature of the language, the rules regarding their use, and the various methods for which they may be used. Various benefits exist to taking this approach.

You cannot anticipate most of the ways in which awk will be used. By learning awk as a language and mastering your understanding of its features, you will reach the goal of this book, quickly determining whether awk can be used to solve a problem and implementing a solution.

If you were to learn awk simply as a utility and not as a language, then if you get an error (especially if learning by example utilities), determining what caused the error is hard. If you understand the language, then by going over the erroneous program, you can more easily determine which statement caused the error, because you understand how each language construct is used to make up the statement. A lot of people complain about the awk error messaging system as being difficult to decipher. My belief is that the problem does not reside in awk's error-messaging system-its run-time error-messaging system is more verbose and informative than GNU C/C++-rather, it resides in not learning the awk language.

Languages are based on very similar principles. Understanding one language can greatly enhance your ability to understand other languages. The reason is that most languages share very similar elements and constructs inherent in the language, such as functions, scoping, coercion, call-by-value, and call-by-reference. (We go over each of these in this book.) If you have already learned a general-purpose language such as C, C++, or Java, then to learn awk will be easier. If you haven't learned a general-purpose language, then the approach taken in this book will help you learn new languages as well as understand this one. In awk, as with most languages, certain design issues go into the creation of the language. One important design issue is orthogonality-which simply means that more than one way of expressing an action must exist. The concept of orthogonality means that more than one language construct (i.e., a for loop and while loop can both be used to implement a program) can be used to solve a program. Orthogonality is beneficial to the programmer because it gives the programmer flexibility in finding more efficient, compact, and creative coding solutions to any given problem using the language. The impact of orthogonality does not mean that all things are considered equal. Although more than one construct can be used to solve a problem, one construct is better to use than another construct in certain programs. Truly mastering a language is the ability to understand when one construct is better to use than another because, as mentioned, it is more efficient (uses less storage or is faster than another construct) or more compact (requires fewer lines of code or is easier to read). This requires practice. My hope is that while reading this book and after, you will practice writing awk programs. Also think about more than one solution to a problem, write an awk program that implements all solutions, and then test which solution is better (more readable, shorter, quicker, and taking less storage space).

I cannot emphasize enough that the best way to learn computer languages and utilities is by practicing. Try out on your own sample queries. In addition, I have found that sometimes the best way to learn is by making your own mistakes. Try figuring out where you went wrong, and try entering a query that you suspect might not work. You might be surprised at the result.

My Intended Audience

This book is not a book on languages. Although this introduction has mentioned language details and issues, whether you have encountered languages before or taken a course on programming languages is not important. Also, you do not need to have an advanced understanding of the UNIX operating system. This book is on grep, sed, and the awk programming language. The book is suitable for novice users of UNIX who have never encountered grep, sed, and awk. It is also suitable for intermediate or advanced users of UNIX who have used grep, sed, and awk but do not have advanced knowledge. All you need to go through this book is a general knowledge of the UNIX operating system. In particular, knowledge of how to execute programs from the command line is necessary.

About the Web Site Companion

This book has a companion Web site, located at:

http://cw.prenhall.com/patsis/

The Web site is composed of various modules designed to augment your exposure to awk, sed, and grep. You'll find a student lounge where you can meet and greet other readers of the Interactive Workbooks and share tips and programs. I have an author's corner, where you can find supplemental material to the book and notes from me about it, errata, and so forth. The answers to the "Test Your Thinking" sections from each chapter of the book have their own module. And additional Self-Review Questions reinforce your understanding of the concepts explored in this book.

Visit the Web site periodically to share and discuss your answers.



Updates

Submit Errata

