InformIT: Mac OS X Unleashed > Intrusion Detection

Table of Contents

Intrusion Detection

In the previous section, you saw some basic methods you can use for securing your machine, including the use of TCP Wrappers. Additionally, we saw that OS X comes with a firewall package, ipfw, which can be used for further securing your machine. In this section, we will look at a couple intrusion detection tools that you can install to further secure your machine, and discuss detecting and reacting to a break-in.

Tripwire

Tripwire is a utility that monitors the integrity of important files or directories. It stores information in a database about files and directories that you have specified. You can then use Tripwire to check whether there have been any changes to your files. It checks the current state of the files against the information in its database. The academic source release of Tripwire 1.3.1 is available from http://www.tripwire.com/. Tripwire Security Systems also has a commercial version of Tripwire, but it might not be available for OS X.

Remember that Tripwire can't detect any unauthorized changes that might have already been made on your system. If you have any doubts about the system's current integrity, you can reinstall the operating system and then install Tripwire.

Although we will not discuss details on compiling Tripwire, here are some tips that might be useful when compiling Tripwire via the ports system described in Chapter 17, "Troubleshooting Software Installs and Compiling and Debugging Manually."

Try /usr/bin/bsdmake if /usr/bin/gnumake does not work.
Compile the dl library.
Whenever you encounter an error that is similar to illegal expression found int, manually run make in the offending directory. You will see many of these in subdirectories of the sigs directory.
If you have not already done so, make a user called man. Follow the same method that you used for making a bin user.
Note that make install moves files when it installs them, as opposed to copying them. If you have to recompile Tripwire for any reason, you also have to manually run make wherever you had to manually run it originally.

If you installed Tripwire via the ports system described in the chapter on additional software packages, /var/adm/tcheck/tw.config was installed on your system. If a typical manual installation does not specifically install a tw.config file for you, the source code does come with samples from which you can work.

To start using Tripwire, edit your tw.config file. The sample configuration file provides details on the syntax of the file. You can specify directories or files for Tripwire to check and what kind of checking it should do in the tw.config file. The basic form of a line in the file is <file> <flags> . The sample tw.config file provides a rather detailed description about the available flags and modifiers to <file> , and the man page provides even more details.

After you feel you have a good, basic tw.config file, you need to initialize the Tripwire database by running tripwire -initialize. In your databases directory, this creates a database called tw.db_ <hostname> . Unless you customized your OS X machine's hostname, this database will likely be called tw.db_localhost.

The documentation suggests that you store your database in a read-only location and that you make a hardcopy of the database contents right away, as added protection. A hardcopy of the database contents will allow you to make a manual comparison, if you become suspicious of the database's integrity. Lastly, the documentation suggests that you might want to generate a set of signatures for the database, the configuration file, and the Tripwire executable by using the siggen utility that is included with the Tripwire package. Make a hardcopy of these as well.

Tripwire has four basic modes: database generation, integrity checking, database update, and interactive. You have already experienced the database generation mode. The integrity checking mode is the mode you are probably most interested in at this point. To run Tripwire in integrity checking mode, simply execute tri p wire. In integrity checking mode, Tripwire checks the integrity of whatever you specified in tw.config and provides a report of what has changed.

Here is a sample of what the output looks like when Tripwire finds no changes:

[localhost:~joray] root# tripwire

     Tripwire(tm) ASR (Academic Source Release) 1.3.1
     File Integrity Assessment Software
      1992, Purdue Research Foundation,  1997, 1999 Tripwire
     Security Systems, Inc. All Rights Reserved. Use Restricted to
     Authorized Licensees.
     ### Phase 1:   Reading configuration file
     ### Phase 2:   Generating file list
     ### Phase 3:   Creating file information database
     ### Phase 4:   Searching for inconsistencies
     ###
     ###                     All files match Tripwire database.  Looks okay!
     ###

Here is a sample of what the outputs looks like when Tripwire finds changes:

[localhost:~joray] root# tripwire

     Tripwire(tm) ASR (Academic Source Release) 1.3.1
     File Integrity Assessment Software
      1992, Purdue Research Foundation,  1997, 1999 Tripwire
     Security Systems, Inc. All Rights Reserved. Use Restricted to
     Authorized Licensees.
     ### Phase 1:   Reading configuration file
     ### Phase 2:   Generating file list
     ### Phase 3:   Creating file information database
     ### Phase 4:   Searching for inconsistencies
     ###
     ###                     Total files scanned:            5528
     ###                           Files added:              0
     ###                           Files deleted:            0
     ###                           Files changed:            1
     ###
     ###                     Total file violations:          1
     ###
     changed: -r-xr-xr-x root            0 8! /bin/ls
     ### Phase 5:   Generating observed/expected pairs for changed files
     ###
     ### Attr        Observed (what it is)         Expected (what it should be)
     ### =========== ============================= =============================
     /bin/ls
           st_mtime: Thu Jun 21 18:28:06 2001      Thu Jun 21 18:22:26 2001
           st_ctime: Thu Jun 21 18:28:06 2001      Thu Jun 21 18:22:26 2001

If you were not expecting any changes, you should be suspicious. If you were expecting changes, there are two ways you can update your database to eliminate the false alarm. You can run tripwire -interactive. This puts Tripwire in interactive mode. Whenever it comes across a discrepancy, it prompts the user whether the database should be updated. You can also update your database by running tripwire -update <filename> , where <fil e name> can be a file or directory. This tells Tripwire to update the specified entry.

To make Tripwire useful, you should run it regularly. The easiest way to do that is to run Tripwire in a daily cron job and have the results mailed to you.

Be sure to read the man pages for tripwire, tw.config and siggen for more detailed information.

PortSentry

PortSentry is a utility available from http://www.psionic.com/. It is part of the Abacus Project, a project by Psionic Software to provide free host-based security and intrusion tools. Psionic Software also makes available LogChecker, which helps you monitor your system logs and HostSentry, which detects anomalous login behavior.

PortSentry monitors connections to ports specified in the portsentry.conf file. If PortSentry detects a connection on one of those ports, you can choose to have PortSentry simply log the connection. You can also configure PortSentry to immediately block the connection. PortSentry adds a deny line for the host to your /etc/hosts.deny or /etc/hosts.allow, depending on which way you are using TCP Wrappers. It then blocks the connection via route or ipfw. You can also provide PortSentry with a list of hosts whose connections it should ignore. You will have to do some testing until you are completely satisfied with your PortSentry configuration.

PortSentry cleanly compiles on OS X, so be sure to read the documentation carefully before you begin. The author clearly outlines the installation procedure step-by-step. Compiling with make generic works fine. By default, the package installs in /usr/local/psionic/portsentry.

The most important file you will work with is portsentry.conf. The first part of the configuration file is the Port Configurations section. Here you specify which TCP and UDP ports are monitored. The author has provided three basic selections: anal, aware, and bare-bones. Of course, you can add any additional ports to whichever set you select.

Next is the Advanced Stealth Scan Detection Options section. Because these options apply only to Linux, you can ignore this section.

The section that follows is the Configuration Files section, where you specify the location of portsentry.ignore, portsentry.history, and portse n try.blocked. The portsentry.ignore file is where you specify which hosts' connections the program should ignore. The portsentry.history file is where PortSentry logs a history of the actions it has taken. The portse n try.blocked file is where PortSentry logs a history of its actions for the current session.

The next section is the Response Options section. In this section, you specify what the automatic response should be for TCP and UDP connections. In the Ignore Options subsection, you specify what level of ignore PortSentry should follow for TCP and UDP connections. You can have PortSentry block scans, not block them, or execute some external command. The Dropping Options subsection is where you select what the blocking response should be. The program can be configured to block via route or via ipfw. The author recommends using ipfw, if you have it running. If you select ipfw, PortSentry, by default, adds a deny rule numbered 1 to ipfw. Of course, you can modify that rule. In the TCP Wrappers subsection, select the correct TCP Wrappers syntax for the way you are using it. In the External Commands subsection, an external command can be specified. In the Scan Trigger Value subsection, you configure the number of port connects that are allowed before an alarm is given. In the Port Banner section, you can specify what text, if any, should be displayed when PortSentry has been tripped.

After you have a basic portsentry.conf file, and you have installed the package, run the following to start PortSentry:

[localhost:~joray] root# /usr/local/psionic/portsentry/portsentry -tcp
[localhost:~joray] root# /usr/local/psionic/portsentry/portsentry -udp

Check /var/log/system.log for the PortSentry startup response. For each PortSentry, you will see some initial startup lines, a line for each port it is monitoring, and a final line indicating that PortSentry is active and listening.

If PortSentry is set to immediately block a connection, here is the type of response you will see in the log:

Jun 21 10:13:56 localhost portsentry[1065]: attackalert: Connect from host:  ryoko.biosci

      
   .ohio-state.edu/140.254.12.240 to TCP port: 21
Jun 21 10:13:56 localhost portsentry[1065]: attackalert: Host 140.254.12.240   has been

      
    blocked via wrappers with string: "ALL: 140.254.12.240"
Jun 21 10:13:56 localhost portsentry[1065]: attackalert: Host 140.254.12.240   has been

      
    blocked via dropped route using command:  "/sbin/ipfw add 1 deny all from 140.254.12.240

      
   :255.255.255.255 to any"

Check your /etc/hosts.deny and run ipfw show. You will see that it does add the offending host to the /etc/hosts.deny file and add an ipfw rule.

If PortSentry is not set to block connections, here is a sample response in the log file:

Jun 21 10:39:56 localhost portsentry[1301]: attackalert: Connect from host: ryoko.biosci

      
   .ohio-state.edu/140.254.12.240 to TCP port: 20
Jun 21 10:39:56 localhost portsentry[1301]: attackalert: Ignoring TCP response per

      
    configuration file setting.

Detecting the Break-in

In spite of all the tools we have looked at so far, it is not necessarily easy to detect a break-in, even with Tripwire. If your machine has been compromised, the intruder might update your Tripwire database for you. The tools certainly help, but they provide no guarantee of protection.

A Linux cluster that we put up a few years ago for a class experienced a break-in within days of being up. That Linux cluster was an out-of-the-box Linux installation and we did not follow the common sense, preventive security measures outlined earlier in this chapter. Unlike OS X, many Linux distributions come with services turned on by default. How did we detect the break-in? Actually, we didn't. The university's security group, which monitors the university's network traffic—among their various duties—told our administration staff that we had a break-in. Not only did they tell us we had a break-in, but they were able to tell us where on our machines to find the culprit software and evidence. Obviously, the security group keeps up-to-date on the latest vulnerabilities and popular cracking software. The break-in was well hidden from us. Common commands such as ls had been replaced and the software was put in a directory called ... that not only usually requires using ls -a rather than ls to see, but their replacement version of ls wouldn't show at all. They also replaced the ps command with one that wouldn't show their sniffing processes, even as the processes were still running, and they tried to delete all other traces of their presence from the machines. A couple of people from the security group came with a floppy that had noncompromised versions of useful commands, and we were indeed able to see this ... directory and the unusual processes that were running.

There are some things you and your facility, if your machine is part of a facility, can do to detect a break-in. Who knows? You might detect one in progress. Your network administrator, who is probably monitoring network traffic, might be able to tell you whether the network is being or was recently probed. This might give you the opportunity to check your logs for any unusual activity. If your machines seem sluggish, someone else could be tying up your resources. Check your logs; check to see who is logged on and check what processes are running. It's possible that you will discover that a normally inactive account has had a recent flourish of activity. Depending on how far along the cracker is your useful commands for detecting unusual activity might have been replaced. Try using ls-F in tcsh, instead of your normal ls, to see whether you can detect the presence of any unusual software or directories. Even if your network administrator hasn't alerted you to any unusual network activity, or even if your machine isn't sluggish, check your logs regularly. You never know what you might find. You might want to consider downloading the LogChecker utility to assist you with monitoring your logs.

Note

The tcsh shell has a built-in ls command named ls-F. Because most cracker wannabes don't know what they're doing, they won't add a fake ls-F, or replace tcsh with a broken version, as well as replace the normal ls command in /usr/bin. Of course, you shouldn't rely on this as a 100% certain way of detecting problem directories or compromised commands. As soon as you trust something, a cracker wannabe with above-average smarts will add it to one of the automated scripts. It's another tool in your arsenal, however, and you need all the tools you can get.

Regularly checking your machine's logs, processes, and so on really is useful. Recently, a machine on our building's network was compromised. How long it had been compromised is unknown. The system administrator noticed an odd ftp process that root was running, but had no reason to be running. When he killed the process, the machine became unusable. When the machine was finally restored, which involved reinstalling the operating system, it was compromised again within two weeks. This time the system administrator noticed that he had four new users that he did not create.

In addition, be especially paranoid around holidays. Crackers know that users won't be around on holidays, and that many system administrators won't be around either. This makes holidays, weekends, and early mornings prime time for crackers because they know they'll have at least a few hours to play before any human is likely to notice them.

Responding to the Break-in

How you respond to a break-in depends on the circumstances surrounding it.

If your security group personnel discovers the break-in, follow their instructions to the letter. What they tell you depends somewhat on the circumstances of the break-in. In the case of a sniffer, they will advise you to secure your machines and have everyone change their passwords. By changing passwords, they mean passwords for any accounts the users on your network might have accessed during the time in question. Yes, that's all passwords on any accounts they might have accessed from your network—including ones on other networks that they've accessed from your machines. However, even in the case of a sniffer, sometimes your security group will advise you to reinstall the operating system. If you have a sniffer, all the traffic that's been across your network has been compromised, including outgoing traffic to other sites, so this might impact machines far beyond yours.

What Is a Sniffer?

A sniffer is an application that watches network traffic as it moves past the sniffer machine. Most modern sniffers can extract usernames, passwords, and other sensitive information from the traffic flow. For an example of a network sniffer for Mac OS X, check out the Ettercap project at http://ettercap.sourceforge.net/. Note: You should never use a sniffer on a network that you do not administer, nor should you activate it without the knowledge of the other users on the network.

If your security group learns that your machine was used for attacking another site, the response they will suggest might surprise you. This time they might not ask you to secure your machine right away. The site that was attacked might request the cooperation of your site in capturing the culprit.

If, however, you detect a break-in before your security group does, you need to decide what the response will be. Do you close down the machine? Do you keep it open to try to catch the culprit? Do you ask your security group what to do? Certainly, there is nothing wrong with asking your security group, but depending on the time of day and number of security problems, you might not be able to find them. Think about how you might respond to a break-in before it actually occurs. If the break-in caused no obvious damage to anyone else's site, swiftly closing down services on your machine and fixing what was done might be the right solution. If damage is done to someone else's site, you might want to keep your machine open for a while in a cooperative effort to catch the culprit. On the other hand, if your site is being used to break into someone else's site and they don't know about it yet, you have the responsibility to stop the attack before the crackers can do any more damage. Cooperation is the name of the game here, and you should behave as reasonably and as professionally as possible. If a remote site requests your help, do everything you can to help—you'd want the same courtesy. If a remote site is being attacked through your site, bend over backwards to limit its damage—you wouldn't want someone to tell you, "Oh, we could have stopped that, but we were in the middle of a game of Quake and didn't want to shut down." Some of this might seem like advice to a system administrator, but remember, you are a system administrator for your OS X machine as well as one of its users.

No matter what you do decide to do, if you have a security group in your company, tell them about the incident and what you did. They need to know everything to best assess your company's security needs.

If your machine might have been compromised, but you can't find any obvious evidence in your logs or some helpful logs aren't available, you might have no recourse but to wipe out your machine and start over. Your users might not like this, but it is better for you to have a machine that you know you can trust, even if that means starting over from scratch. Your users probably prefer the downtime to the potential loss of their data. Again, tell your security group folks what you've done. In all likelihood, they will support your decision to wipe out your machine and start over from scratch. They would rather have you do that if you have any doubt about your machine, than have you let even one suspect machine stay on the network. Roughly one-third of our campus network and a related government site were taken down for several days a while ago when someone persisted in allowing one compromised machine to stay on the university's network. Fortunately, our machines weren't involved in that one.

Given that one day you may have to wipe your machine in response to an attack, there are some common sense measures you can take to make restoring your data easier:

Keep a copy of your OS X CD in a location where you can find it.
Keep your software CDs and registration numbers in a location where you can find them.
Archive to CD-R any additional third-party software you downloaded and installed, especially if it was difficult to compile or is important to you.
Archive to CD-R any data you don't want to lose.

Yes, that's it. Don't lose your OS X CD and software CDs. Back up any additional software you compiled and your data. That seems too obvious, doesn't it? It should be, but even the best of us need that reminder. Remember the machine on our network that was compromised twice in one month? The system administrator killed just one process, but the machine was hosed as a result. They probably spent at least $5,000 for their machine. They also lost a software package that was the key to a faculty member's research, but did not have a backup copy of it anywhere. After investing in the latest hardware, you can't afford to not spend a few dollars more and a little bit of time to preserve your data. With the current cost of CD-Rs, you should even be able to afford good CD-Rs for your data. In other words, if your machine has been compromised, it is best to be prepared to have the smoothest recovery possible. Backups and your original software CDs should help with this.

Where to Go from Here | Next Section

Previous Section