Home > Articles > Operating Systems, Server > Microsoft Servers

  • Print
  • + Share This

Services and Protocols

Now that you have had a look at the OSI model, the following section examines the Windows NT network environment's specific services and protocols and how they relate to this model.

Relationship to OSI Model

The Windows NT network environment is extremely complex with respect to the types and ranges of services and protocols supported. Figure 3.7 illustrates just how complicated this environment is. The intent here is not to present an exhaustive set of services and protocols in the Windows NT environment, but rather to provide a basic understanding concerning where within the OSI model some of the most fundamental and widely used protocols typically fit. Note also that the level at which a given protocol is utilized depends on the particular implementation based on that protocol, as explained shortly. In other words, one developer may implement a specific layer in a certain protocol, while another may implement the same funtionality in a different protocol.

Figure 3.7
Some major services and protocols in Windows NT networking.

Layer 3 includes protocols such as IP and Internetwork Packet Exchange (IPX)—a mainstay protocol for Novell NetWare until release 5.0, but also the same protocol that runs in the Windows NT networking environment for interoperability with legacy NetWare hosts. Layer 4 includes the Transmission Control Protocol (TCP), the UDP, the NetBEUI, and Sequenced Packet Exchange (SPX) protocol (another protocol that in the Windows NT networking environment allows interoperability with NetWare hosts).

Relevance to Security

Although each layer presents its own set of challenges so far as security is concerned, certain layers potentially pose more security-related risk than do others. The physical layer, for example, is the layer vulnerable to packet-capture attacks in which network traffic is captured by a physical or logical (that is, a program) device. Attacks, such as IP spoofing, capitalize on weaknesses in implementations of programs that utilize the IP protocol, a Layer 3 protocol. The UDP, a Layer 4 protocol, is transmitted without provision for notification of the sending host in case the transmission fails or contains one or more errors. In contrast, another Layer 4 protocol, TCP, is a considerably more reliable protocol in that it provides a mechanism for notifying a sending host when something goes wrong with a TCP-based transmission. Therefore the UDP is, in general, more vulnerable to certain types of network attacks than is TCP.

IP Spoofing

IP spoofing is a type of network attack that can be directed against virtually any machine that processes IP traffic. The goal of an IP spoofing attack is to establish a connection between a client unknown to a server by making that client appear to be a legitimate client, and then to exploit a relationship between the server and the bogus client to gain unauthorized access. Here is a well-known way to perpetrate an IP spoofing attack:

  1. Make the legitimate client unable to respond to the target server. This can be done by using a utility that "wedges" the legitimate client's ports—making the service or daemon that receives input from each port wait for input that will never come, thereby making the machine unresponsive to other inputs, such as connection request acknowledgements from other servers. This step is necessary because if the legitimate client were able to respond to the target server, the bogus client would not be able to "break in" to their communication.

  2. Send a SYN packet from the bogus client to the target server to request that a connection be opened. This packet must indicate that the connection request is from the legitimate client (for example, must bear the IP address of the legitimate client), even though in reality the packet must originate from the bogus client. The bogus client's request packet includes the initial sequence number (ISN) for that client.

  3. The target server sends a SYN packet to the legitimate client containing data such as the server's ISN in addition to the client's ISN incremented by one. The legitimate client's ports are wedged, however, so the legitimate client will never respond to this packet.

  4. The connection request is dropped if the client does not increment both ISNs (that is, for both the client and the server) and increment them in a manner that the server expects. The software running on the bogus client must then send a reply SYN packet containing the source address of the legitimate client with appropriate ISNs (one for the client and one for the server) incremented by one. Deriving the client's ISN is easy; this ISN is in fact an arbitrary number of which the server is not initially aware. The challenge is guessing the target server's ISN. The best clue concerning the value of the ISN the server has sent to the legitimate client in the first place is within the contents of already captured network traffic; packet dumps can reveal the previous ISNs for the server's connections to other systems. If the target server's ISN for a connection request from an entirely different client began with 24080 a few seconds previously and the ISN is always incremented by one for any new connection request, for example, the next ISN for a new connection is likely to be 24081 if the initial ISN is not random. IP spoofing software that returns a ISN of 24082 from the bogus client to the server would therefore be very likely to correctly anticipate the appropriate ISN.

  5. If the bogus client sends the correctly incremented value of both ISNs to the server, the attacker will have established a connection between the two. The attacker can then attempt to exploit a relationship between the two machines to gain unauthorized access to the target server. Windows NT 4.0, for example, supports the (remote shell) rsh command that can allow trusted access from one machine to another without requiring that a password be entered.

Although predicted by Steve Bellovin in "Security Problems in the TCP/IP Protocol Suite," (ACM Computer Communications Review, Vol. 19, Issue 2, 1989, pp 32–48), the first reported IP spoofing attack was not observed until late 1994. For several years afterward, IP spoofing was one of the most frequently observed types of attack on the Internet. IP spoofing is not now as commonly reported as a few years ago, although it still poses a potentially major threat to organizations, in that so many automated IP spoofing tools are so widely available. The best (albeit not infallible) countermeasure is deploying a firewall or screening router that blocks all incoming packets that indicate they originated from a host within the network protected by the firewall or screening router. This measure prevents spoofing attacks originating from outside one's network, but does not prevent such attacks if they are initiated from within the same network.2

Although most observed IP spoofing attacks have targeted UNIX systems (in particular, implementations based on Berkeley Standard Distribution [BSD] UNIX), Windows NT is also vulnerable to these attacks. Unless Service Pack 63 for Windows NT 4.0 is installed on a given Windows NT host, that host will (under many connection contexts) linearly increment the server's ISN from one connection to the next in a predictable manner, making the machine extremely vulnerable to IP spoofing attacks. Service Pack 4 for Windows NT 4.0 causes a server that receives an IP connection request to generate a reasonably random ISN in the SYN packet it returns to the requesting client, virtually precluding the possibility of IP spoofing. Several ways exist to verify that this service pack has been installed in 4.0 systems. One way to do this is to bring up the Command Prompt and enter winver.

If your Windows NT installation includes Service Pack 4 or up, you will obtain the following type of output from an About Windows NT dialog box (see Figure 3.8).

Figure 3.8
User Panel.

Another way you can determine that latest service pack installed is to invoke a Registry Editor, and then check the value of the following key:

    Hive: HKEY_LOCAL_MACHINE

    Key: Software\Microsoft\WindowsNT\CurrentVersion\

    Value: CSDVersion

Another way to determine the service pack level on the local host, as well as hosts across the networks, is to run the SPQuery utility from MTE Software (http://www.mtesoft.com). It lists the numbers of installed service packs as standard screen output.

Finally, another way to prevent IP spoofing attacks is to use a firewall or router with ACLs to reject traffic coming into a network that bears the source IP address of any internal host. Any incoming traffic with an internal source address is almost certainly bogus (and could indeed indicate an IP spoofing attack); legitimate incoming traffic should bear the source IP address of some external host.

About the Protocols Themselves

The Windows NT networking environment includes a wide variety of protocols, virtually all of which affect security one way or another. What are some of these protocols? Where do they fit into the OSI Model? This section explores these important issues. Figure 3.7, which shows some (but certainly not all) of these protocols, provides an initial overview. Consider the following protocols.

  • Remote Procedure Call (RPC). RPC is a UDP-based protocol used in setting up communications, such as negotiating the particular ports to be used in establishing a connection.

  • Network File System (NFS). NFS is a protocol (generally based on UDP) for file sharing that enables a user to connect to remote disks as if connecting to the local machine. Several Windows NT–based NFS implementations (typically based on the SMB protocol) are currently available.

  • Named pipes. Named pipes are mechanisms that provide a direct channel to services that support applications. They are advantageous because they allow programs to reach them by referring to a name instead of requiring that a full path be specified.

  • File Transfer Protocol (FTP). FTP is a TCP-based protocol for establishing sessions in which files are transferred between computers.

  • Trivial File Transfer Protocol (TFTP). TFTP is similar to FTP, although it is not a connection-oriented protocol. Based on UDP, it is used to download fonts and configuration files to hosts that broadcast their needs. Because it does not confirm whether data are actually being sent to their destination, it involves less overhead to create and maintain connections than does FTP.

  • Simple Mail Transport Protocol (SMTP). SMTP is the protocol that establishes the structure of Internet mail through a special syntax. It also defines the conventions for setting up SMTP connections, transmitting both the sender and receiver's addresses, and sending the subject and main body of mail messages.4

  • Server Message Block (SMB). SMB is an implementation of redirectors. Redirectors handle client requests for access to remote resources on a drive with a shared directory or another type device (for example, a printer) by taking these requests and reformatting them according to the needs of the protocols that will process these requests. Finally, SMB forwards the requests to either a lower- or higher-level protocol.

  • NetWare Core Protocol (NCP). NCP is a protocol implemented in Novell NetWare. NCP is used in the Windows NT networking environment for access to remote resources on NetWare machines.

  • Network Basic Input Output System (NetBIOS). NetBIOS is an Application Programming Interface (API) used in the Windows environment to provide session-layer connectivity between machines.

  • Winsock. Winsock is not a protocol; it is instead a socket (a combination of a service and port) used by APIs for client applications regardless of the underlying protocol. It is commonly used to provide network-based access to Windows applications.

  • Telnet. Telnet is a protocol that furnishes a command-line interface for emulating a virtual terminal on a remote computer. This in turn enables users to interact with a remote computer.

  • Network Basic End User Interface (NetBEUI). NetBEUI implements the transport layer and is only used in smaller, local networks because of the fact that it is unroutable. All things considered, NetBEUI is a relatively fast protocol with low overhead.

  • Transmission Control Protocol (TCP). TCP is the most common transport protocol used today. It is a connection-oriented transport layer protocol that verifies packets sent by one machine (the source machine) arrive at the destination machine. TCP thus establishes a virtual connection between two machines.

  • User Datagram Protocol (UDP). UDP is a connectionless protocol that does not have built-in guaranteed delivery. One of its main advantages is that it involves less overhead than connection-oriented protocols such as TCP.

  • Sequenced Packet Exchange (SPX). SPX is a transport layer protocol used primarily in contexts in which Windows NT hosts access Novell NetWare computers.

  • Internet Protocol (IP). In many respects, IP is described as the protocol that provides the underlying functionality of nearly all higher-layer Internet protocols. Its functionality includes addressing (which includes checking packet headers to determine whether the information therein is correct), fragmentation (in case a router received packets that are too large to handle), and determining each packet's time-to-live (which, if expired, causes the packet to be discarded). IP handles each packet completely independently of any other packets sent over the network.

  • Internetwork Packet Exchange (IPX). IPX, a routable protocol, is a very fast and highly established protocol, but it cannot be used on the Internet because it supports a different addressing convention from the one used by the IP (see the following section). Novell developed IPX/SPX for use in NetWare, but Microsoft has developed its own implementation of this protocol, the NWLink protocol. This protocol is completely compatible with Novell's IPX/SPX implementation.

The default protocol in most Windows NT network applications is a protocol suite called NetBIOS over TCP/IP (NBT). Numerous problems with NBT have emerged over the years due to dependencies between the NetBIOS and TCP layers of networking, in addition to other reasons. Performance and denial-of-service problems have resulted. As you will see shortly in the section titled "NetBIOS and SMB-Based Vulnerabilities," the NetBIOS layer is also filled with perils and pitfalls, among the more notable of which are dependence on primitive lookup mechanisms to retrieve NetBIOS name data and the capability to crash remote hosts by sending malformed NetBIOS packets or packets with illegal parameters. These and many similar vulnerabilities are discussed in more detail throughout this chapter.

Note again that the protocols described in this section by no means constitute the full range of protocols in the Windows NT networking environment. These protocols are some of the most commonly found ones that also often pose the most serious security-related threats. They are collectivel, only a portion of the possible protocols found within the Windows NT networking environment.


  1. IP spoofing is most often initiated by external clients to deceive internal servers that these clients are internal.

  2. Actually, this was changed in SP4, not SP6. However, after SP6 it was discovered that there was a bug in the ISN generation. Therefore a new hotfix was issued. It is described in Q243835, and is available for SP4-6.

  3. While there are SMTP implementations for NT, it is not a standard protocol.

  • + Share This
  • 🔖 Save To Your Account

Related Resources

There are currently no related titles. Please check back later.