The Dirty Work: Maintaining Basic Linux Systems Security
This article is derived from /etc/hosts, Jon's column in Web Hosting magazine.
So it's time to secure your system. You've marshaled your resources to allow enough time to get it done, you have equipment, and you know what you're trying to secure and why. But what are the first steps you should take to secure a system?
Briefly, the top six steps to secure a Linux (or Unix) system are:
- Keep current backups.
- Disable unnecessary services.
- Keep system patches current.
- Limit access to resources.
- Avoid cleartext protocols.
- Automate what you can.
Before we can examine these points individually, we need to take a step back and look at how systems are cracked in the first place.
If we intend to keep crackers out of our systems, we have to know how they get into them. Most cracked boxes I've seen have two things in common: too many unused services running, and too few updates or patches installed. Both of these are errors in administration, and are avoidable with just a little bit of knowledge.
Recently, I spent a portion of my time cleaning up after a hacker attack on a Linux box. It wasn't a Linux box that I ran; to protect its identity, let's call the box, "Spike," and its system administrator, "Pete." Spike was a Red Hat Linux 6.0 box.
What generally happens is this: An admin like Pete installs the system with every option turned on. "Well, I don't know what all of this stuff is," he might say, "so I'd better install all of it now, just in case." Because Pete is new to the admin game, it doesn't occur to him that every service adds to the number of potential security holes on his system. Because Pete also doesn't read Bugtraq or keep up with his distribution's security announcement mailing list, he doesn't realize when one of those programs he's got running is now exploitable, so he doesn't patch the system.
Thousands upon thousands of "script kiddies"—wannabe crackers with no discernible talent—are constantly scanning as many computers attached to the Internet as they can. Sooner or later, one of them will happen upon Spike and realize that there's an "exploit," a pre-packaged script that will open up a latent security hole, which will run on that system. Moments later, that script kiddie has root privileges on Spike.
Now, let’s take a more in-depth look at the top six steps to secure a Linux (or Unix) system.
1. Keep Current Backups
It's a mistake to think that these things only happen to other admins, especially only to admins less knowledgeable than ourselves. No security is ever perfect, and it's always possible that a master hacker has decided to break into your system using his own skills and exploiting holes not yet publicly known.
The only reasonable preventative technique is to keep current backups, preferably two weeks to two months' worth of backups for your system. That way, once you detect an attack, you can figure out what's been changed and recover what's been deleted.
Backups are also an important issue with regard to physical security: What happens to your critical files if your machines are stolen, or if your building burns to the ground? Look at off-site backups for any files that are business-critical.
Whoever hacked Pete's machine installed a "rootkit," a set of replacement utilities that can hide changes to the system—for example, a replacement for 'ls' that hides specific files in directory listings. It's notoriously difficult to clean up after a rootkit has been installed, and it's best just to copy known-safe data files onto a freshly installed system. Without backups, it's impossible to be truly confident about the safety of a system.
2. Disable Unnecessary Services
New exploits will be found. Fewer things are more certain in the computer security world than the constant development of new exploits. The best way to limit your exposure to unknown exploits is to not run software you don't need. If you discover that you need the package, it's always relatively easy to install it later. Spike was running DNS services, even though we were already providing DNS and even though it wasn't properly configured and thus was doing no good.
Recently, a "back door" was reported for Red Hat's "piranha" utility, which provides basic clustering services. However, the back door turned out to be a default password, and piranha can't be used until that password is changed. Nevertheless, dozens or hundreds of machines were broken into through this method. Because removing this hole is necessary for the software to be useful, it's clear that large numbers of people are running software that they're not using. Don't be one of them.
The best way to disable a service is to remove it entirely from your system. Most systems use package management software to handle installing and removing software. The two most popular package managers are RPM (the Red Hat Package Manager) and dpkg (the Debian package management system). RPM is used not only by Red Hat Linux, but also by SuSE Linux and others; dpkg is used by Debian Linux, Storm Linux, and Corel Linux, among others.
If you're using a system based on the RPM package manager, you can find out what package contains a given executable by running the command rpm -qf /path/to/program/I/want/to/remove. The q option is for query, and f indicates that you want to query the database for a given file. RPM will return the name of the package that owns this file, if it was installed via RPM. The package name will look something like insecure_garbage-1.0-1, and you can then remove the package with rpm -e insecure_garbage.
If your system uses the Debian package management system, you can locate the package that owns the offending program by running dpkg --search /pth/to/program/I/want/to/remove. After this, you can remove the package with the command dpkg -r insecure_garbage.
Remember, if it's not installed, it can't be exploited!
3. Keep System Patches Current
Even though Pete had installed the name server on Spike, he might have been protected if he had also updated his system: As near as I can tell, Spike was cracked in the middle of April—but Red Hat had a fix for the problem announced in the middle of last November, making the hole at least six months old. This may sound extremely lax of Pete, but I've seen machines broken into through holes that had fixes available more than a year earlier.
Sometimes it's difficult to know that you need to update your software, but most of the time, vendors have email lists you can subscribe to that will notify you of security patches. Certainly, virtually all vendors have Web pages with the appropriate information. If your Linux distribution doesn't have such a Web page, I strongly advise switching systems as soon as this is feasible. In any case, joining the appropriate mailing list is a must to keep your system updated.
NOTE: Linux Distribution-Specific Security Mailing Lists
Red Hat Linux
4. Limit Access to Resources
Limit physical access to your systems to necessary personnel: Your data is not safe if a thief can walk off with your server, or if any guy in a brown UPS jumpsuit can walk out of your building with a pile of hard drives. Make sure you have policies regarding physical access, and that those policies are followed. If your server is not safely locked away, or in a monitored machine room, you run a great risk: Given physical access to the machine, any halfway-competent sysadmin or cracker can gain root access in minutes, and might install a back door on your system.
You can limit access to resources on the software side of things, too. Linux Pluggable Authentication Modules (PAM) allow you to control the maximum size of files on your system, the maximum number of processes a user can run, the maximum amount of memory a user can allocate, and so on. If you want to prevent a denial-of-service attack, configuring these resources is critical. PAM also lets you control which users can log on at what times, and what sort of authentication is required to do so: If you want to allow your users to log in only from nine to five—that is, if you have no interest in their accomplishing useful work—you could set PAM to do so. PAM handles different configurations for users and groups of users, so you can give your sysadmins more flexibility than your janitors, if you so desire.
More information on Linux PAM is available at http://www.kernel.org/pub/linux/libs/pam/.
Firewalls may be your friend as well. Although a firewall is only a tool for implementing a policy, a strong implementation of a strong policy is a good addition to your security structure. Your default policy, of course, should be to deny any sort of connection. You can then make exceptions for specific traffic: Allow any DNS connections from offsite to my DNS servers, allow anyone at all to send Web requests to my Web servers, and so on.
If you're not rich enough to throw hardware at the problem, most modern Linux distributions (that is, those based on the 2.2 series of kernels) have IP Chains, a stateless firewall solution. It's not as effective as a real firewall, but it's quick to set up and will do a pretty good job of things. More information on IP Chains is available at http://www.linuxdoc.org/HOWTO/IPCHAINS-HOWTO.html.
5. Avoid Cleartext Protocols
Any time you send passwords in the clear over your network, you're asking for trouble. Any service you provide to users outside of your network that depends on sending cleartext passwords over your network is asking for trouble. Any questions?
Unfortunately, it's not that simple: telnet, rlogin, rsh, FTP, POP3, and IMAP4 all use cleartext passwords. (IMAP4 does not always require cleartext passwords, but it does support them.) If you offer mail accounts to your users, they will demand POP3 and IMAP4. Although SSL-encrypted versions of those protocols are available, not everyone has a client capable of using the secure protocol, so you will have to provide the insecure protocol as well, subjecting those users to possible password-sniffing attacks. But minimizing your exposure is a good thing: Any time that you can choose not to send passwords unencrypted over the network, you should choose to do so. Install the SSL-enabled IMAP and POP daemons; not all your users will avail themselves of them, but it can't hurt.
Whenever possible, use SSH instead of telnet, rlogin, rsh, rcp, and FTP. SSH is a replacement for all of those protocols (with the partial exception of FTP) that automatically encrypts your connection, protecting you from password sniffing. The official version of SSH is commercial software; their Web site is http://www.ssh.fi/.
OpenSSH, a freeware clone of SSH, is now available as well. Because of SSH's odd license, I highly recommend using OpenSSH wherever possible; the OpenSSH Web site is at http://www.openssh.com/.
Anthony Santoni, administrator at Orisis.net—a Linux-based Maryland Web hosting company—says that SSH has protected his users. He says, "I have seen systems on shared networks compromised by sniffers relatively easily. The only accounts that were not acquired were those users connecting via SSH. It is so very easy for a person on a shared network to pull down a sniffer and just wait for the passwords to come in." Of course, having a switched network is no protection if your mail server, or any other generally used service, is compromised, Santoni notes, "On a switched network(i.e., one that is not contention-based), sniffing is a little harder. You have to compromise one machine first, then set the sniffer up there to catch passwords going out to other servers."
6. Automate What You Can
Wherever possible, choose not to do the dirty work yourself. Anthony Santoni says, "With any online business, the name of the game is automation. Good admins and developers do their very best not to do the same work over and over again. If you can automate tasks, you can then move on to the next issue on your plate. In reference to Web development, automation through CGI, mailing lists, and dynamic content is necessary for increased hits and customer convenience. The more you automate, the more resources you free up for other work." I couldn't have said it any better myself.