How Slow Can you Go?
This event concerns a remotely monitored site that had poor response time on a good day. One day while attempting to examine activity, the response time became painfully slow. It was so slow, you could type in one character and it would take about 30 seconds to see it echoed back on the screen. This was pretty annoying, but signaled that the site had some issues other than normal poor response time.
Although this was occurring, we were collecting a copy of their Shadow sensor data at our site. In an attempt to explain the poor response time, the site's Shadow events of interest were examined. It showed that they were getting a lot of fragmented activity directed at their network address of 192.168.133.0 (this is a translated address for anonymity purposes). Upon further examination, it was discovered that this had been going on for many hours. Here is a sample of the records that they were getting:
12:01:12.150572 dos.com > 192.168.133.0: (frag 54050:1480@4440+) 12:01:17.560572 dos.com > 192.168.133.0: (frag 54050:1480@2960+) 12:01:17.570572 dos.com > 192.168.133.0: (frag 54050:1480@4440+) 12:01:22.200572 dos.com > 192.168.133.0: (frag 54050:1480@1480+) 12:01:22.210572 dos.com > 192.168.133.0: (frag 54050:1480@2960+) 12:01:22.220572 dos.com > 192.168.133.0: (frag 54050:1480@4440+) 12:01:22.230572 dos.com > 192.168.133.0: (frag 54050:1480@5920+) 12:01:27.240572 dos.com > 192.168.133.0: (frag 54050:1480@2960+) 12:01:27.250572 dos.com > 192.168.133.0: (frag 54050:1480@5920+) 12:01:37.230572 dos.com > 192.168.133.0: (frag 54050:1480@1480+) 12:01:37.240572 dos.com > 192.168.133.0: (frag 54050:1480@2960+) 12:01:37.240572 dos.com > 192.168.133.0: (frag 54050:1480@4440+) 12:01:37.250572 dos.com > 192.168.133.0: (frag 54050:1480@5920+) 12:01:42.300572 dos.com > 192.168.133.0: (frag 54050:1480@1480+)
You see dos.com sending fragmented packets to the network address. As mentioned, this activity had been happening for several hours. There are a couple of problems with the traffic that need to be examined. See if you can find the three problems associated with fragmentation in the previous TCPdump output.
First, a normal fragmented packet train usually has two or more parts:
There is an initial fragment that has an offset of 0 and the More Fragments flag set (+):
Recall that the fragment format is as follows:
frag FRAG-ID:BYTES-IN-CURRENT-FRAGMENT@OFFSET-INTO-FRAGMENT-DATA [+]
There might be intermediate fragments that are neither the first nor last fragments. An intermediate fragment has a non-zero offset and the More Fragments flag set. The + flag indicates that the more fragments bit is set or there is another fragment following the one being sent. The More Fragments flag is set in the first and intermediate fragments.
There is a final fragment, one in which the more fragments bit is not set: no + flag.
This activity appeared on Shadow's hourly wrap-up from the default because both the fragmentation and the destination address having a final octet of 0 (the network address 192.168.133.0).
The fragmentation that is seen in this log has some definite problems:
There is no first fragmentone that has an offset of 0.
You see repeated offsets for fragments that are in the same fragment train with the fragment ID of 54050. For instance, the fragment offset 4440 is repeated several times.
There is no final fragmentone that doesn't have the More Fragments flag (+) set.
It is possible that the offset values are not chronological because the fragments don't necessarily arrive in the order in which they were sent.
After doing some research about the topology of the remote site, we discovered that our sensor was located behind (inside) a packet-filtering device that blocked inbound ICMP echo requests. That is the reason we believe that the initial fragment was never seen. Keep in mind that only the first fragment in the fragment train carries the embedded protocol header, such as the information to say that these packets were associated with ICMP echo requests. We can only surmise that the fragmented activity was associated with the dropped ICMP echo requests.
The packet-filtering device that blocked this activity was a router that did not keep track of state. Therefore, it blocked the first fragment of the fragment train because it was the one that contained the information that this was an ICMP echo request. The router had no means of associating the first fragment with subsequent ones. It appears obvious to us that the subsequent packets all share the same fragment ID and are assumed to be associated with the blocked one. Yet, this router did not maintain that information and allowed the subsequent fragments into the network.
However, this doesn't explain why no final fragment was observed. This should have nothing to do with a router that is incapable of keeping track of state. The only explanation for not receiving a final fragment is that is was intentionally omitted.
Normally, fragments are reassembled by the destination host only and not by intermediate routers through which they travel. However, in this case, the router attempts to reassemble the fragmented packets because they are sent to the network address 192.168.133.0 on which the router resides. This particular router has an old Berkeley Software Distribution (BSD) TCP/IP style stack that responds to this "broadcast" so that it attempts to reassemble the fragments.
The router has limited cache for reassembly. The combination of the repetition of the same fragment ID, the more fragments bit set in every fragment, and the missing first and last fragments severely encumbered the router so that it couldn't do routing work. The router would never time out on reassembly of these packets because it kept seeing evidence that more fragments were coming. This was a successful denial of service (DoS) against the router. When the hostile IP was blocked on an external router, the response time returned to normal.
Why didn't this router expire the incomplete set of fragments with an ICMP "IP reassembly time exceeded" message? After all, isn't this a prime candidate for resource exhaustion, waiting for a fragment or fragments that are never sent? The problem is that for the "IP reassembly time exceeded" message to be delivered and for the receiving host to expire the fragments, the first fragment must be received. Because the outermost router blocked these, the first fragment never arrived, and others could not be expired.
Although some routers block incoming ICMP echo requests, denial of service attacks against the router should not occur for "normal" traffic. The DoS attack succeeded against this particular router because of the broadcast address, the repeated fragment ID, and the missing fragments. After the problem was discovered, the activity was blocked from the hostile source IP address. This blocked all inbound traffic including fragments because the IP address is repeated in each of the IP headers of every fragment.
This was successful, and the response time returned to its normal slow (but not painful) state. The attackers must have sensed this; chances are that the monitored site must have foolishly sent ICMP errors that indicated that their activity was blocked. The attackers responded by attempting the same attack with a different source IP address on the same subnet.
Explanation Acknowledgement and Additional Reference
Many thanks and much credit to Vicki Irwin of the SANS Institute for her assistance in figuring out the router DoS. She referenced the following for a discussion of this and similar exploits: