The SMTP, POP, and IMAP Protocols
This content is excerpted from Chapter 35 of TCP/IP Unleashed by Karanjit Siyan (Sams Publishing: ISBN 0672323516).
This chapter is from the book
This chapter is from the book
This chapter is from the book
The Internet standard for e-mail is the Simple Mail Transport Protocol (SMTP). SMTP is the application-level protocol that handles message services over TCP/IP networks. SMTP was defined in 1982 by the Internet Engineering Task Force (IETF) and is currently specified in RFCs 821 and 822. SMTP uses TCP port 25.
Although SMTP is the most prevalent of the e-mail protocols, it lacks some of the rich features of X.400. A primary weakness of standard SMTP is the lack of support for non-text messages.
Besides SMTP, there are two other protocols for delivering mail to workstations. These are POP3 and IMAP4.
MIME and SMTP
MIME (Multipurpose Internet Mail Extensions) supplements SMTP and allows the encapsulation of multimedia (non-text) messages inside of a standard SMTP message. MIME uses Base64 encoding to convert complex files into ASCII.
MIME is a relatively new standard, and although it is supported by almost all UA applications at this time, there might be a chance that your e-mail application does not support MIME. If that is the case, you will likely use one of the other encoding methods (BinHex or uuencode) described later in this chapter.
MIME is described in RFCs 2045–2049.
S/MIME
A new specification for MIME exists that allows it to support encrypted messages. S/MIME is based on RSA's public key encryption technology and helps prevent messages from being intercepted and forged.
RSA Public Key/Private Key Authentication
Short for Rivest, Shamir, and Adelman, the inventors of the algorithm, RSA provides public key/private key encryption. The basic idea is that data encrypted with the public key can only be decrypted with a private key. With S/MIME, the sending UA encrypts a random string of data using the public key of the receiving (remote) user or UA. The recipient UA then decrypts the message using the private key.
For more information on RSA, refer to http://www.rsa.com/.
S/MIME is currently specified in RFCs 2311 and 2312.
Other Encoding Standards
Several other standards exist for encoding non-ASCII messages. The more popular of these are BinHex and uuencode.
BinHex stands for Binary Hexadecimal and is considered by some to be a Macintosh version of MIME. Uuencode stands for UNIX-to-UNIX Encoding because of its UNIX origin, although it is now supported by many non-UNIX platforms. Although MIME, uuencode, and BinHex do have several fundamental differences, they accomplish the same primary goal—allowing non-text files to be sent in text messages. The method you use will depend upon your mail UA and the UAs used by your target recipients. Fortunately, most modern UAs take care of the encoding and decoding for us.
SMTP Commands
Part of the simplicity of SMTP is that it uses a small number of commands. Table 35.2 lists these commands.
Table 35.2—SMTP Commands as Specified in RFC 821
|
Command |
Description |
|
HELO |
Hello. Used to identify the sender to the receiver. This command must accompany the hostname of the sending host. In the extended protocol (ESTMP), the command EHLO is used instead. See the "Extended SMTP" section later in the chapter for more information. |
|
|
Initiates a mail transaction. Arguments include the "from" field or the sender of the mail. |
|
RCPT |
Identifies the recipient of the message. |
|
DATA |
Announces the beginning of the actual mail data (the body of the message). The data can contain any 128-bit ASCII code and is terminated with a single line containing a period (.). |
|
RSET |
Aborts (resets) the current transaction. |
|
VRFY |
Used to confirm a recipient user. |
|
NOOP |
This "no operation" command specifies no action. |
|
QUIT |
Closes the connection. |
|
SEND |
Lets the receiving host know that the message must be sent to another terminal. |
|
The following commands are specified, but not required, by RFC 821: |
|
|
SOML |
Send or mail. Tells the receiving host that the message must be sent to other terminals or mailboxes. |
|
SAML |
Send and mail. Tells the receiving host that the message must be sent to other terminals and mailboxes. |
|
EXPN |
Used to expand a mailing list. |
|
HELP |
Requests helpful information from the receiving host. |
|
TURN |
Requests that the receiving host take on the role of the sending host. |
SMTP command syntax is simple as well, as you can see in the following SMTP example:
220 receivingdomain.com ñ Server ESMTP Sendmail 8.8.8+Sun/8.8.8; Fri, 30 Jul 1999 09:23:01 HELO host.sendingdomain.com 250 receivingdomain.com Hello host, pleased to meet you. MAIL FROM: 250 Ö Sender ok. RCPT TO: 250 Ö Recipient ok. DATA 354 Enter mail, end with a ì.î on a line by itself Here goes the message. . 250 Message accepted for delivery QUIT 221 Goodbye host.sendingdomain.com
The resulting mail message would look something like:
From username@sendingdomain.com Fri Jul 30 09:23:39 1999 Date: Fri, 30 Jul 1999 09:23:15 -0400 (EDT) From: username@sendingdomain.com Message-Id: Content-Length: 23 Here goes the message.
SMTP Status Codes
When a sending MTA issues SMTP commands to the receiving MTA, the receiving MTA responds with special status codes to let the sender know what is happening. Table 35.3 lists the SMTP reply codes as specified in RFC 821. These codes are grouped by status, as defined by the first digit in the code (5xx for failure, 4xx for temporary problem, 1xx–3xx for success).
Table 35.3—SMTP Reply Codes
|
Code |
Description |
|
211 |
Help reply, system status |
|
214 |
Help message |
|
220 |
Service ready |
|
221 |
Closing connection |
|
250 |
Requested action okay |
|
251 |
User not local, forwarding to |
|
354 |
Start mail input |
|
421 |
Service not available |
|
450 |
Action not taken, mailbox busy |
|
451 |
Action aborted, local error |
|
452 |
Action not taken, insufficient storage |
|
500 |
Command unrecognized or syntax error |
|
501 |
Syntax error in parameters or arguments |
|
502 |
Command not supported |
|
503 |
Bad sequence of commands (given out of order) |
|
504 |
Command parameter not supported |
|
550 |
Action not taken, mailbox unavailable |
|
551 |
Not a local user |
|
552 |
Aborted: Exceeded storage allocation |
|
553 |
Action not taken, mailbox name not allowed |
|
554 |
Transaction failed |
The numeric codes are defined in the RFC. However, the accompanying text, while suggested in the RFC, is left up to the postmasters and MTA administrators to define. Sometimes they get a little creative.
Extended SMTP
SMTP has proven itself to be a strong, useful e-mail protocol. However, there is a recognized need for extensions to standard SMTP. RFC 1869 spells out a means by which extensions can be added to SMTP. It does not list specific extensions, but rather provides a framework for the addition of necessary commands. An example is the SIZE command. This extension allows a receiving host to limit the size of incoming messages. Without ESMTP this would not be possible.
When a system connects to an MTA, it can provide the extended version of the HELO command, EHLO. If the MTA supports extended SMTP (ESMTP), it will respond with a list of commands it will support. If it does not support ESMTP, it provides an error (500 Command not recognized) and the sending host reverts back to SMTP. The following is a sample ESMTP transaction:
220 esmtpdomain.com ñ Server ESMTP Sendmail 8.8.8+Sun/8.8.8; Thu, 22 Jul 1999 09:43:01 EHLO host.sendingdomain.com 250-mail.esmtpdomain.com Hello host, pleased to meet you 250-EXPN 250-VERB 250-8BITMIME 250-SIZE 250-DSN 250-ONEX 250-ETRN 250-XUSR 250 HELP QUIT 221 Goodbye host.sendingdomain.com
Table 35.4 describes the common ESMTP commands.
Table 35.4—Common ESMTP Commands
|
Command |
Description |
|
EHLO |
Extended version of HELO |
|
8BITMIME |
Indicates 8-bit MIME transport |
|
SIZE |
Used to specify the size limit of the message |
Examining SMTP Headers
You can learn a wealth of information by closely examining the headers of your SMTP messages. Not only can you see whom the message is from, the subject, the date sent, and the intended recipient, you can see every stopping point made by the message en route to your mailbox. RFC 822 specifies that the header contain, at a minimum, the sender (From), the date, and a recipient (TO, CC, or BCC).
NOTE
Technically, TO and CC are identical. CC (Carbon Copy) is a historical term that dates back to a time when everything was typed on typewriters and carbon paper was used to produce duplicates.
BCC (Blind Carbon Copy)
The Received header allows you to see everywhere a message has been prior to arriving at your inbox. It can be a great troubleshooting tool. Consider the following example:
From someone@mydomain.COM Sat Jul 31 11:33:00 1999 Received: from host1.mydomain.com by host2.mydomain.com (8.8.8+Sun/8.8.8) with ESMTP id LAA21968 for ; Sat, 31 Jul 1999 11:33:00 -0400 (EDT) Received: by host1.mydomain.com with Internet Mail Service (5.0.1460.8) id ; Sat, 31 Jul 1999 11:34:39 -0400 Message-ID: From: "Your Friend" To: "'jamisonn@host2.mydomain.com'" Subject: Hello There Date: Sat, 31 Jul 1999 11:34:36 -0400
In this example, you can see that a message was sent from someone@mydomain.com. From mydomain.com, the message was delivered to host1. That message was then received by host2 from host1, where it was delivered to me. At each stop along the way, the receiving host is required to add its header, which must include a date/time stamp. It is interesting to note that in the preceding example, there is a discrepancy in timestamps. Host2 (my computer) reports that it received the message at 11:33:00. Host1 reports that it received the message at 11:34:36, over a minute after I received the message. This is due to a lack of clock synchronization between the two hosts.
Advantages and Disadvantages of SMTP
Like X.400, SMTP has several primary advantages and disadvantages.
The advantages are as follows:
SMTP is very popular.
It is supported on many platforms by many vendors.
SMTP has low implementation and administration costs.
SMTP has a simple addressing scheme.
The disadvantages are as follows:
SMTP lacks certain types of functions.
SMTP lacks the security specified in X.400.
Its simplicity limits its usefulness.
Client Mail Retrieval with POP and IMAP
In the early days of Internet e-mail, users were required to log in to their e-mail server and read their messages there. Mail programs were usually text-based, and lacked the user-friendliness that many users were used to. To solve this problem, some protocols were developed that enable you to have your mail messages delivered directly to your computer desktop. These UA retrieval protocols also come in very handy when a user "roams," or works at several different computers.
Two widely used methods are Post Office Protocol (POP) and Internet Mail Access Protocol (IMAP).
The Post Office Protocol (POP)
POP allows local mail UAs to connect to the MTA and pull mail down to your local computer, where you can read and respond to the messages. POP was first defined in 1984, then updated by POP2 in 1988. The current standard is POP3.
POP3 UAs connect via TCP/IP to the server (typically port 110). The UA enters a username and password (either stored internally for convenience or entered each time by the user for stronger security). After authorized, the UA can issue POP3 commands to retrieve and delete mail.
POP3 is a receive-only protocol. POP3 UAs use SMTP to send mail to the server.
POP3 is defined by RFC 1939.
Table 35.5 lists the POP3 commands.
Table 35.5—POP3 Commands
|
Command |
Description |
|
USER |
Specifies the username |
|
PASS |
Specifies the password |
|
STAT |
Requests the mailbox status (number of messages, size of messages) |
|
LIST |
Lists an index of the messages |
|
RETR |
Retrieves the specified messages |
|
DELE |
Deletes the specified messages |
|
NOOP |
Does nothing |
|
RSET |
Undeletes messages (rollback) |
|
QUIT |
Commits changes and disconnects |
The Internet Mail Access Protocol (IMAP)
POP3 is a very good and simple protocol for retrieving messages to your UA. However, its simplicity results in a lack of several desired features. For instance, POP3 only works in offline mode, meaning that the messages are downloaded to the UA and deleted from the server.
NOTE
Some implementations of POP3 support a "pseudo-online" mode that allows the messages to be left on the server.
The Internet Mail Access Protocol (IMAP) picks up where POP3 leaves off. IMAP was first conceived in 1986 at Stanford University. IMAP2 was first implemented in 1987. IMAP4, the current specification, was accepted as an Internet standard in 1994. IMAP4 is currently specified in RFC 2060. IMAP4 is found at TCP port 143.
Table 35.6 lists the IMAP4 commands as specified in RFC 2060.
Table 35.6—IMAP4 Commands
|
Command |
Description |
|
CAPABILITY |
Requests a list of supported functionality |
|
AUTHENTICATE |
Specifies an authentication mechanism |
|
LOGIN |
Provides username and password |
|
SELECT |
Specifies the mailbox |
|
EXAMINE |
Specifies mailbox in read-only mode |
|
CREATE |
Creates a mailbox |
|
DELETE |
Deletes a mailbox |
|
RENAME |
Renames a mailbox |
|
SUBSCRIBE |
Adds mailbox to active list |
|
UNSUBSCRIBE |
Removes mailbox from active list |
|
LIST |
Lists mailboxes |
|
LSUB |
Lists subscribed mailboxes |
|
STATUS |
Requests mailbox status (number of messages, and so on) |
|
APPEND |
Adds a message to the mailbox |
|
CHECK |
Requests a mailbox checkpoint |
|
CLOSE |
Commits deletions and closes mailbox |
|
EXPUNGE |
Commits deletions |
|
SEARCH |
Searches mailbox for messages meeting specified criteria |
|
FETCH |
Fetches parts of a specified message |
|
STORE |
Changes data of specified messages |
|
COPY |
Copies message to another mailbox |
|
NOOP |
Does nothing |
|
LOGOUT |
Closes the connection |
POP3 Versus IMAP4
There are many fundamental differences between POP3 and IMAP4. Depending on your UA, your MTA, and your needs, you might use one or the other, or even both. The advantages of POP3 are
It is very simple.
It is widely supported.
Due to its simplicity, POP3 is often limited. For example, it can only support one mailbox, and the messages must be deleted from the server (although many implementations support a "pseudo-online" mode allowing messages to be left on the server).
IMAP4 has several distinct advantages:
Stronger authentication
Support for multiple mailboxes
Greater support for online, offline, or disconnected modes of operation
IMAP4's online mode support allows UAs to download only a subset of the messages from the server, search for and download only messages matching a certain criteria, and so on. IMAP4 also allows a user or UA to move messages between server folders and delete certain messages. IMAP4 is much better suited for the mobile user who needs to work at several different computers, or the user who needs to access and maintain several different mailboxes.
The major drawback to IMAP4 today is that it is not widely deployed by ISPs, notwithstanding the fact that many IMAP4 clients and servers exist today.