Product Author Bios

Thomas Boutell is the author of the World-Wide Web Frequently AskedQuestions list, a FAQ that is distributed to all Web-related newsgroups. He is also the author of the leading CGIlibrary written in C, as well as the gd dynamic graphics library.

0201422190AB04062001

The simple, static hypertext documents that currently dominate the Web canconvey a great deal of information, but eventually their limitations becomeclear. What if you wish to provide dynamic data--information that changes overtime? What if you want to sell products on your Web site and secure paymentinformation from users? Or what if you seek to provide a search facility thatpermits a Web database to be explored? Dynamic resources of this sort areaccomplished through CGI (Common Gateway Interface) programming.

CGI programs can take advantage of any resource available to the servercomputer to generate their output and can also accept input from the userthrough forms. These two basic capabilities have led to a wide variety ofapplications, such as forms processing, generation of inline images and movies,the formatting of data sets based on queries to a database, real-time updatesto Web pages, and more.

CGI Programming in C and Perl shows you how to create theseinteractive, multimedia documents via CGI programming in two practicallanguages: C, which has distinct performance advantages, and Perl, one of themost popular for CGI today. Applications and source code are presented in both languages.

You'll learn how to:

• generate HTML pages and images on the fly

• get CGI access on your ISP's site

• ensure security for your CGI-activated site

• parse form submissions directly

• send e-mail via forms and CGI.

Supplements

The CGI Standard

The CGI standard is intended to provide a consistent interface between Web servers and programs that extend their capabilities. As discussed in Chapter 2, Web servers are able to deliver static documents from files without the assistance of other programs. However, dynamic content, database interfaces, and responsiveness to user input are all desirable and cannot be accomplished by static documents alone.

This chapter gives an overview of the CGI standard, examining both its definition and its reasons for existence. Topics introduced in this chapter are covered in more detail and with full examples in later chapters. So keep this in mind if you find portions of the chapter difficult to follow.

The Need for a Standard

Dynamic documents have been around almost as long as the Web itself. From the first days of the Web, programmers have found ways to generate Web pages on the fly. Early systems of this sort often involved custom servers--servers developed solely to deliver a particular type of dynamic content. The simplicity of the original HTTP1 (HyperText Transfer Protocol) specification made this approach practical, and it is sometimes still employed today when efficiency is of paramount concern.

As Web servers gained in sophistication, it soon became apparent that it was not practical to replicate the advanced security, logging, communications, and load-management features of a modern server for every application that required dynamic documents. So Web server developers began to provide interfaces through which external programs could be executed, usually in response to a request for a document in a designated portion of the document tree. The NCSA httpd (HTTP daemon) was the first Web server to provide such an interface.2

These first external-program interfaces represented a significant advance but did not solve the standardization problem. The various Web servers followed separate standards for the development of external programs, so it was impossible to write a single external program that would be useful with all servers.

The Goals of CGI

The CGI standard was developed jointly by NCSA (the National Center for Supercomputing Applications) and CERN (the European Laboratory for Particle Physics) in response to the need for a consistent external-program interface.3 In addition to a consistent, standard interface, the CGI standard seeks to provide reasonable guarantees that user input, particularly form submissions, will not be lost due to the limitations of the server operating system. The CGI standard also attempts to provide the external program with as much information as possible about the server and the browser, in addition to any information that may be known about the user. Finally, the CGI standard attempts to be straightforward so as to make the development of simple CGI applications easy.

Largely, the CGI standard has achieved these goals. Other emerging approaches are sometimes more efficient, especially when tuned to specific operating systems (see Chapter 15, "What's Next: CGI and Beyond"), but the portability and simplicity of the CGI standard has led to its widespread use.

CGI does not always achieve its goal of simplicity. The parsing of form data is somewhat complex, especially when done properly and thoroughly. However, its popularity has led to the development of several freely available tools to simplify the task of CGI programming, including several Perl libraries and the cgic library for C (see Chapter 8).

CGI and the HyperText Transfer Protocol

The server executes CGI programs in order to satisfy particular requests from the browser. The browser then expresses its requests using the HyperText Transfer Protocol.1 The server responds by delivering either a document, an error status code, and/or a redirection to a new URL according to the rules of that protocol. While CGI programs usually need not speak the HTTP directly, certain aspects of it are important to CGI.

HTTP requests can be of several types, which are called methods. For CGI programmers, the most important are GET and POST.

The GET method is used to request a document and does not normally submit other input from the user. If the requested URL points to a CGI program, the CGI program generates either a new document, an error code, or a redirection to another document (see "CGI Standard Output" later in this chapter). The CGI program can identify this situation by examining the REQUEST_METHOD environment variable, which in this case will contain the string GET.

In some cases, there is a certain amount of user input, which the browser submits as part of the URL. This may happen for example when the <ISINDEX> HTML tag is used or an HTML form delivers its data via the GET method. This information will then be stored in the QUERY_STRING environment variable. This approach is rather unwieldy and so should be avoided in new CGI programs.

The POST method is used to deliver information from the browser to the server. In most cases, the information sent is a form submission. The CGI program then reads the submitted information from standard input (see "CGI Standard Input" later in this chapter for the format of the data). In this case, the REQUEST_METHOD environment variable will be set to the string POST. To be certain the posted information is a form submission, the CGI prgram examines the CONTENT_TYPE environment variable to make sure it contains the string x-www-form-urlencoded. The latest Web browsers can also post other forms of data to the Web server.

After examining the submission, the program typically generates either a new document, an error code, or a redirection to another URL (see "CGI Standard Output" later in this chapter).

Other methods, such as PUT, are coming into use, but GET and POST remain the most important HTTP methods for CGI programming.

CGI Environment Variables

The HTTP daemon (server) shares a variety of information with the CGI program. This information is communicated in the form of environment variables.

In C, environment variables are normally accessed using the getenv() function. For example, the following code example retrieves the PATH_INFO environment variable. This variable contains any additional information in the URL of the user's request beyond the path of the CGI program itself and preceding a question mark, if any:

char *serverName;

serverName = getenv("PATH_INFO");

In Perl, environment variables are found in the %ENV associative array. The following Perl expression also refers to the PATH_INFO environment variable:

$serverName = $ENV{'PATH_INFO'} ;

Appendix 1 contains a complete list of the CGI environment variables. The most important are introduced as they are used in the next several chapters.

CGI Standard Output

CGI programs are normally expected to output one of three things: a valid Web document, a status code, or a redirection to another URL.

Outputting a New Document

To output a new document, a CGI program must first indicate the type of data contained in the document. For instance, the following content type header indicates that the data that follows is in HTML:

Content-type: text/html

VERY IMPORTANT: TWO line breaks must follow the Content-type line. Otherwise, your document will be ignored. A blank line signals the end of the headers and the beginning of the document, if any.

Following the content type header, the CGI program can output any valid HTML document. Many examples of HTML documents are in this book, beginning in Chapter 4.

Information about other content types, such as image/gif, is presented in Chapter 10 and Appendix 2.

Returning a Status Code

If an error occurs, the CGI program may wish to output a status code rather than a document. In this case, the following MIME header whould be output:

Status: #### Status Message

where #### is replaced with any valid HTTP status code4 and Error Message with an explanatory message for the user.

IMPORTANT: The status line must be followed by two line breaks. Otherwise, the status code will be ignored.

No further output is necessary.

Redirecting the Browser to Another URL

It is not uncommon for a programmer to decide that a request is best handled by redirecting the user to another URL. The other URL can be on the same or another system. In the first case, the server will usually intervene and attempt to resolve the request without sending the redirection back to the browser. In the second case, the new URL is transmitted back to the browser, which then makes a connection to a new server.

To redirect the browser to another URL, output the following header:

Location: URL

where URL is replaced with any valid URL. If the URL refers to a document on the same server, http://hostname can be omitted, and performance gains are realized by doing so. To ensure compatibility with all servers, you are advised to begin such local URLs from the document root (begin with a / character). Note: Some browsers also require the header URI:URL... in addition to Location:URL... .

IMPORTANT: The status line must be followed by TWO line breaks. Otherwise, the redirection will be ignored.

No further output is necessary. More information about this subject is presented in Chapter 12.

Perl and C code to handle output from CGI programs is presented throughout this book beginning in Chapter 4.

CGI Standard Input

Earlier external-program standards specified form data to be stored entirely in environment variables or passed on the command line, an approach that risks truncation under some operating systems when the amount of information is too great. While the CGI standard allows this approach, it also allows and encourages an approach in which the form data is passed to the external program as its standard input, meaning the data can be accessed with simple standard I/O calls such as the C functions getc() and fread().

When no data has been submitted by the user or a form has been submitted with the GET method, standard input will contain no information. However, when data has been sent by the user using the POST method, the data will appear at standard input. Use of the POST method is the preferred method for form submissions. The format for form data is:

keyword=value&keyword=value&keyword=value

that is, a stream of keyword and value pairs, where keywords and values are separated by equal signs and pairs by & signs.

IMPORTANT: Note that all characters that are escaped in URLs are also escaped in posted form data. That is, if characters such as &, =, and % are typed by the user and submitted as part of the request, they will appear escaped in standard input as follows:

%xx

where the ASCII value of the character in hex is substituted for xx. For instance, the %, ASCII value 37 decimal, 21 hexadecimal, appears as follows:

%21

The CONTENT_LENGTH environment variable will always contain the total number of characters of data to be read from standard input.

Perl and C code to handle the CGI standard input stream is presented in Chapter 7, "Handling User Input: Interacting with Forms."

Conclusion

In this chapter, I introduced the basic concepts of CGI. In Chapter 3, we begin writing CGI programs.

References

1. W3 Consortium, "HyperText Transfer Protocol Specification."
[URL:http://www.w3.org/hypertext/WWW/Protocols/Overview.html]

2. NCSA HTTPd Development Team, "Upgrading NCSA httpd."
[URL:http://hoohoo.ncsa.uiuc.edu/docs/Upgrade.html]

3. W3 Consortium, "Overview of CGI."
[URL:http://www.w3.org/hypertext/WWW/CGI/Overview.html]

CD Contents

Download the CD Contents from the book CGI Programming in C and Perl

Table of Contents

(Each chapter ends with a Conclusion.)

Acknowledgments.


1. World Wide Web Documents.