Inspecting Your Firewall
It's not easy to inspect firewalls. The firewall sits there on the rack, guardian against the Internet's hordes. Each day there will be a lot of access requests to approve, implement, or review. How do you start? I'll outline some general steps and observations that have helped me to review several companies' firewalls.
Step 1: Understand the Environment
You must limit the scope of your investigation to make sure that you'll be able to complete the work. Why is the review being done? Who's sponsoring the review? If you can answer these two questions decisively, you can keep the scope of your review manageable. For example, have you been commissioned specifically to review firewall rules? Or are you doing a rule evaluation during a security investigationin which case, you may need to reach out to devices beyond traditional firewalls? (Such devices may include access points brought by peers to the control firewalls.)
Some places have two different types of firewalls guarding their doorways. Some have one type of firewall but a lot of doorways, equaling several firewalls to review. Some have one demilitarized zone (DMZ); some have several layers and firewalls in their DMZ. Some even consider proxy servers (reverse and forward) to be firewalls. Are remote-access gateways "firewalls" if they implement filtering?
Sometimes the questions seem innocent: "Is this rule set fairly secure?" To answer that question, however, you may need to find out why some connections don't go through the firewall.
Get the scope of the investigation determined quickly, and document it for approval by the review's sponsor. Then get network technical diagrams and list the firewalls whose rules you will inspect. Complex environments have multiple firewalls, with some merely routing firewalled traffic to another inspection point. Those types of flows and relationships must be recorded in writing and understood by the entire team.
Now that you know the purpose and devices of the evaluation, you can go forward.
Be sure to resist limitations that keep you from completing your review. When your credibility is involved, you may need to keep an evaluation open beyond the company's use of the vague term secure. Does the organization assume that connections outbound from the corporate intranet are harmless? If so, they don't understand the nature of inside-out attacks caused by spyware. Do you think the business rules seem okay but the organization's approval process is shaky? That's in scope for your review process. When reviewing firewalls, how you do something is as important as what's been done.
Step 2: Check the Network
Firewalls are very dependent on network management. For some reason, large companies often have firewall management isolated from other networking tasks. If firewall management is outsourced, there may be total isolation from network-management efforts, which can create interesting challenges (I'll get to that topic later in this article). Before examining the firewall rules, examine the firewall setup. Begin by checking the network on several key points:
IP address allocation
IP address translations
Let's look at some specific issues.
IP Address Allocation
Find the group most responsible for IP address managementtypically a router-management group that must handle network address translations (NAT) and all internal and external routes. Determine just which network addresses are considered intranet, DMZ, supplier, and so on. Firewalls must reflect the network layout and the assumed trust that the organization gives to each domain, whether internal or external. Once you have the exact class C's that are considered intranet, for example, research the translations done at the organization's borders.
IP Address Translations
Find out which internal IP addresses are translated to which exact IP addressesor even a single address if port address translation (PAT) is used. You need to know which IP addresses should appear in which domain (intranet, Internet, etc.), including the true IP address of those IP addresses that seem internal.
Work with the router group to determine the organization's default route. In other words, where do packets go if they have no set route or path in the organization?
Applying the Information
You may wonder why a firewall inspection has you tracking network layout and routing architecture. Go to several servers at random and check their connection tables and routes, typically by using the NETSTAT and ROUTE commands. Do you see IP addresses that aren't in your list of authorized internal IP addresses? Check the logs for key DHCP and DNS servers in your organization. Are external domains and IP addresses indicated in the logs? Check ARP tables on key routers. Get into the background records that list IP addresses in use in the organization, and see whether those addresses are authorized.
If you don't have containment, checking firewall rules is pointless. Your security gateway is not as secure as it should be; there are simply too many paths around the firewall. The rule set is not the final determiner of your organization's security. If you're in the middle of a security investigation, prioritize your precious time to other tasks if the firewall setup is sound.
What could cause leakage across the border? Remember that new corporate acquisition with an independent connection to the Internet? How do they secure their access? Do they have second-tier connections to their suppliers? If so, what translations are done at their entry points? What translations are done at your base organization's remote-access points? Maybe one of those is hacked. Does the default gateway flow through the firewall? If not, there's no outbound inspection. Some intruder may be doing a packet-interjection attack against one of your generic routing encapsulation (GRE) tunnels on the Internet. This is one example why you as a firewall person must be involved in routing decisions; if two routers create an unencrypted GRE tunnel between two Internet endpoints, without encryption, the traffic can be scanned. Attackers can then interject encapsulated traffic from an external attack system into your tunnel, sometimes merely by faking the source IP address of a router. The traffic comes in, attacks the server, and then flows out the default routethe uninspected default routeback to the attacker's computer.
More causes of intranet perimeter compromise: Can users connect to their home computers via various tunneling/VPN technologies? If the device in your network is configured to route/forward/bridge traffic, then traffic from the user's home PC and network will come into your network. Perhaps the user has an unsecured wireless access point, or offers "free high-speed Internet access" via your Internet connection and the user's wireless access point.
How about some examples? I read an Internet blog by a disgruntled employee who had opened his home access point to anyone. He had a permanent VPN tunnel to his employer; people who were "cool about it" could use the employer's bandwidth to surf the Net. Another article discussed the massive hacking benefits of getting employment as a janitor. Once employed, planting a wireless access point in a conference room made great sense to the author. The live network port and the large windows in the conference room built at the edge of the building meant that the signal would carry for miles with the right equipment. Then there are articles on building high-strength antennae from chip cans.
Uncontrolled wireless is uncontrolled access to your network and weakening of your security gateway. By the way, attaching wirelessly often provides internal addresses for the attack machine. In fact, unauthorized IP addresses could be the result of someone who has contracted for a private high-speed connection from your facility to the Internet. Private connections can provide a high-speed line to the facility, with no involvement with any networking group. Do you control OSPF at your facility? If not, that private line can soon become the preferred route with the cheapest metrics. This is what happened at one hospital when a doctor ordered a private line. Medical information avoided the encryption routers and took the more direct, unencrypted, private route to the Internet.
Maybe someone isn't aware of your NAT policy and has never applied NAT on a connection. If you have a lot of CISCO routers, one of the administrators may be using a little-known feature called "Proxy Arp," which allows a router to forward a packet that normally cannot find the path to a destination.
In short, there are many ways in which unauthorized connectivity can occur.
Step 3: Check the Baseline Configuration
Fundamentally, a firewall is intended to be a security device, controlling connectivity. Surprisingly, many organizations assume that someone whose role is routing or switching can do a great job with firewalls. Yes, the firewall will route packets. Yes, it can participate in routing designs. Yes, some vendors have products that can combine basic firewalls and elegant policy-based routing. But the firewall may inadvertently route more than it should. Consider the following examples of traffic that must be blocked or controlled by firewalls.
Case 1: Fragmented Traffic
As mentioned earlier, several attacks divide payload among packet fragments whose pieces overlap in key bytes. As the packets are gathered and reassembled at the server, the payload is reconstructed and the attack ensues. Somewhere in the connectivity chain between a high-risk and lower-risk domain, there must be a device that does packet reassembly and checking. If your firewall can't do this, it's time to find another firewall.
Case 2: IP Source-Routed Traffic
Yes, you read this heading correctly. No, I'm not talking about SNA source-routed traffic. A key TCP/IP strength is dynamic routing of traffic; doesn't the dynamic nature of IP routing seem to contradict the term source-routed?
Let me explain.
Early use of IP wasn't as effortless as it is now. Early router failures caused problems. IP designers created a way to specify the path of routers in the packet header. Strict source routing required the packet to go to each IP address in the list, and only the IP addresses in the list. Loose source routing simply had the packet going to each address in the list. The result of this forgotten and seldom-documented capability? Maybe your worst intruder has a tool that can fake the source IP address tied to a packet. The source address is much like the return address on a postcard; why would you fake it? Because some server applications authenticate a command by the source IP address, assuming that a packet can only come from the device documented in the source IP address header.
The intruder can fake a source IP address, but can't send the packet. Here's why: If he or she sends the packet, the return traffic goes to the real instance of the IP address. Then the real device at the IP address issues a RESET command, and this command alerts the target server that something bad is going on. This alert often stops and tears down the connection. But let's see what happens with source routing. The intruder specifies the attack PC in the source routing list of addresses and sends the packet with the fake IP address. The packet goes to the server and, by IP convention, the server constructs the return traffic with the routing information added to the packet. On the way back to the source IP in the header, the packet goes to a few IP addresses (to confuse law enforcement) before it goes to the attack computer. The attack computer sets up the next packet in the exchange, sends it, and then dumps the return traffic it received. The return traffic never reaches the authentic device at the IP address.
Case 3: ICMP Redirect
Many people feel warm and fuzzy about the security that their switched network brings. Switches are different from other Layer 3 devices. Long ago, individual computers were plugged into a hub. The hub distributed all traffic to all devices on the hub. This meant that an intruder could put a computer's networking card into a special mode (PROMISCuous) and then review and record all traffic to all devices. This also meant that all devices had to share available bandwidth (bandwidth being a measure of network capacity). Too many devices at one connectivity point soon created a traffic jam that had a lot of packets colliding and a lot of devices resending missed or lost traffic.
Switches differ from hubs; these devices set up individual connections. At no time are devices sharing a connectivity point, so the conversation between two devices is not easily viewed by a third device. It's a lot like running a private network line between two devices.
That is, until the intruder uses an ICMP Redirect command. This command instructs the target computer to use a different IP address for routing functions. Computers can't possibly know all the paths or routes to all destination computers. Instead, they have a gateway settinga router that makes sure that the sent packet makes it to the next router and ultimately to the destination. The switching setup ensures that all traffic gets a dedicated pathway between the two endpoints that cannot be monitored. But with an ICMP Redirect command, the gateway setting forces the traffic to go one level higher, through a device that the intruder choosesan attack PC that will record and review the traffic for good information or for IDs and passwords.
Case 4: Spoofing
As mentioned earlier, firewalls can introduce a concept of "state"interrogating traffic trying to come into your intranet to see whether the traffic is authorized. This setup can prevent a lot of incidental network attacks by dropping traffic that's clearly out of context. What connectivity does the Internet need to your internal servers, for example? Perhaps the ability to send a packet to the CHARGEN service, thereby starting one of the attacks mentioned earlier? By recognizing the source of the traffic, you can prevent problems early.
So how do we recognize traffic and determine its context? Most firewalls create the concept of "state" by defining each interface and the general types of IP addresses that are common to the interface. For example, the interface closest to your intranet should have the intranet IP ranges associated with it. Once this is configured, the firewall can associate traffic IP addresses with security domains, such as DMZ versus intranet.
Applying the Information
One of the first things that must be done with the firewall is to configure context. Certain types of traffic must be dropped, which requires certain baseline configurations to be established and updated consistentlybefore the first rule is applied! For example, source-routed traffic is no longer needed and typically represents an attempt to deliver traffic without the normal routing restrictions. Does the firewall drop source-routed traffic by default? A firewall must reassemble packets and either check them or let them go to a downstream checker. ICMP messages and codes must be controlled; aside from the Redirect message discussed earlier, there are messages that confirm that incoming traffic was dropped for administrative reasons. If those messages are allowed to pass, an intruder can verify the firewall's control rules. Therefore, ICMP messages must be restricted to the barest set needed for managing connectivity. These restrictions must work in both directions. In fact, given the inside-out nature of some attacks, firewall inspections are needed in both directions. Either the firewall or some other device must implement these controls.
A more difficult problem is one of context. IP ranges are associated with interfaces to prevent spoofing. Good firewall design stops traffic that's out of context. Early security designs used simple filters that had no concept of context. Imagine a filtering device that dumps all connections initiated from a DMZa common security practice. If a DMZ computer is compromised, you might think that the intranet is connected. After all, any traffic from the DMZ will be filtered at the gateway. But what if the source IP address is spoofed to be an internal address? After all, several interesting applications use the connectionless UDP protocol. If the packet approaches the DMZ interface while acting as if it's from the intranet itself, what will your firewall do? Maybe your organization has carefully configured each interface such that if traffic claiming to come from the intranet hits the DMZ interface, the DMZ interface drops it.
Think you're safe? You're not.
RFC 1918 outlines several address ranges that are reserved for private use within an organization. If faking internal addresses doesn't work, intruders often try addresses in RFC 1918 and RFC 3330. Believing that the Internet will drop these packets, many administrators don't anticipate these addresses showing up at the Internet interface, and therefore don't configure the interface to drop these packets.
Is your firewall configured with default deny or default permit? Some security systems deny all inbound traffic by default and only allow it if permitted. Some take the opposite approach, which can let unconsidered traffic into an organization.
Everyone is asked to create filters against these addresses on their exterior gateways, but few do. Are these important RFC 3330/RFC 1918 address ranges filtered at the Internet firewall interface? If not, and it isn't done somewhere else in the organization's connectivity path from the Internet, your evaluation should point out this failing.
But the work doesn't end here. Large organizations experience many changes in their networks. Redirectors allow organizations to use private RFC 3330 addresses in a DMZ for servers. While the redirectors front-end the servers, sometimes developers ask for a back connection into the intranet to get data. This configuration may hurt your filtering and create an opportunity for intruders. You'll see these private addresses attempting to cross the line you set up to stop attacks from untrusted domains. Similarly, organizations redeploy IP addresses registered to them. The addresses are taken from the intranet into the DMZ to support the growth of web applications. What happens when one of these DMZ servers is compromised? Here again, an out-of-date firewall will sense that the DMZ server's traffic comes from the intranet itself. Without anti-spoofing rules in place, many firewalls pass attack traffic from the DMZ directly into the intranet. After all, how many firewall administrators think to track network changes and then to update anti-spoofing rules?
Has your company sold a division recently? Are those addresses registered to that division still considered part of the organization's intranet? What happens if firewall administration is outsourced, and the outsourcer never gets news of new address space or address space redeployments?
Application designs and firewall settings must be configured to drop traffic that's out of context for the interface. DMZ interfaces must drop traffic claiming to come from the intranet. Internet interfaces must drop traffic claiming to come from RFC addresses. Intranet interfaces must drop traffic that claims to come from non-internal addresses, to contain many inside-out attacks.
Consider the "sense" of the firewall. Review its placement in the organization's networks. What kind of traffic are you expecting at each interface or doorway to the firewall? If your assumptions are too broad, you may get surprise traffic at the firewall that takes advantage of what you don't anticipate. Think outside of the box. Be creative. What would you do to sneak traffic past the firewall's design?
Step 4: Review Advanced Security Features
Firewalls take a lot of maintenance and complete understanding of security hacks and obscure networking protocols. Major manufacturers have created easily selected settings that do a lot of this work for you. For example, some firewall code can run on server platforms. In these instances, there is a time delta during boot between the interface activation and the firewall controlling it. Smart administrators select the firewall feature that disables forwarding until the firewall controls the interface.
A second key feature that many firewalls implement is protection against SYN attacks. Earlier, this article discussed attacks that attempt to use up a system's TCP connections, keeping the machine busy and making it unusable to other users. Several firewall vendors have systems in place to detect these SYN attacks and to prevent them from reaching servers susceptible to this attack.
In addition to these basic abilities, current firewalls have increased ability to detect and prevent attacks staged against applications. Such features often are offered as subscription services, much like antivirus tools, which enables security filters to be updated quickly and dynamically. This capability makes the organization able to withstand increasingly complex attacks while relieving administrators from having to create complex filters by scratch. If such advanced features are not enabled or even tested, what can you say in defense of that firewall implementation? Are the rules still worth evaluating when so many basic security elements are unused?