Introducing OpenBSD 3.9
Date: Apr 28, 2006
OpenBSD began life as a fork of NetBSD, the oldest of the currently active BSD projects. A personality clash between Theo de Raadt and the rest of the NetBSD team lead to Theo’s access to the project’s CVS tree being revoked.
At the time, the standard way of developing F/OSS was for a small group of developers to have access to a private CVS repository. They would then publish periodic snapshots of the tree for download. One of Theo’s first acts after leaving NetBSD was to create a public anonymous CVS server containing his local copy of the NetBSD tree, including the improvements he had not been allowed to commit to the main tree.
This transparency in the development process gave the new project the name OpenBSD. The openness is reflected in the project’s license philosophy. Most of the base system is BSD licensed, with a few MIT licensed or public domain pieces. A few GPL’d components, such as the GNU Compiler Collection, are permitted but only in cases where no alternative is available with a more permissive license. Closed-source code is not permitted in the base system at all.
Another feature of this openness, and something that sets OpenBSD apart from many other systems, is the documentation. In OpenBSD, the UNIX manual pages are considered authoritative. If a program or function call does not behave exactly as the manual describes, this is considered a bug. This is reflected in the development process, which does not allow any code that result in a user-visible change to be committed to the tree without an accompanying update to the documentation.
Within the last few years, OpenBSD has become more militant in its advocacy of openness. Prior to the 3.7 release, the team decided that its reverse-engineered drivers for several Adaptec cards did not meet the required quality for inclusion in the base system, and could not without documentation from the manufacturers. When this documentation was not provided, they removed support for Adaptec RAID hardware for the 3.7 release.
OpenBSD is also very unfriendly to the idea of drivers that rely on binary blobs; lumps of closed-source code that are loaded by an "open" driver. The NVIDIA drivers for Linux are an example of this; they provide a thin skeleton of open source code that is compiled and linked against the kernel which then loads the real, binary driver. Because the code for the real driver is not open, it cannot go through the same auditing process as the rest of the kernel, and so are likely to introduce bugs and security holes.
Some manufacturers are very reluctant to release the specifications required to create a driver. A common reason for this is that their product has a large number of bugs that are worked around by the driver. In some cases, it is because they heavily optimize a specific code-path used by a benchmark (sometimes even to the point of compromising accuracy) in order to gain a marketing advantage over their competitors.
The most common excuse given is that the documentation or the drivers contain valuable IP that they are not allowed to release, or do not wish to provide to their competitors. While this may be true of some complex graphics card drivers that include a significant portion of an OpenGL implementation in software, it is not true of the specification from which the drivers were created. To quote Marco Peereboom, one of the OpenBSD developers:
"One fills in a few structures, one hands it off to the hardware, one gets notified when the work is done. There that’s all a driver does. There is no IP in filling out a structure and handing it off to hardware."
Another example of OpenBSD’s activism came from the license restrictions placed on the firmware for wireless network cards. To save costs, most WiFi cards do not include their firmware in ROM. Instead, it is loaded by the driver at runtime. This means, obviously, that a working driver must have a copy of the card’s firmware. Unfortunately, most manufacturers placed redistribution restrictions on these firmware files, meaning that the end user had to download them directly from the manufacturer, rather than having them shipped with OpenBSD. Since then, most Southeast Asian manufacturers have modified their licenses to allow redistribution of the firmware.
In 2004, Theo De Raadt was presented with the Free Software Foundation’s Award for the Advancement of Free Software in recognition of these efforts.
Security
The forked NetBSD code base that OpenBSD began with had a significant amount of history. It inherited code dating back to the first BSD releases in the 1970s. Some of this had never properly been audited or even read by anyone other than the original author. As such, it was full of small bugs, many of which were potential security holes.
In 1996, shortly after OpenBSD began, they started auditing the codebase. This is an ongoing process, and continues to this day. Every time a new type of bug is discovered, the entire codebase is searched to find other occurrences of it.
In spite of the audit process, the OpenBSD team admits that it (along with third-party developers) is not infallible. The OpenBSD kernel contains a number of features that protect against various categories of attack. More information about these can be found in the Security section of the OpenBSD web site.
What’s New in 3.9?
OpenBSD has followed a regular release schedule since the project’s creation. The next release, 3.9, is due on May 1, 2006.
To get an idea of what’s new in the 3.9 release, I asked some of the OpenBSD developers. Before getting on to the new features, I asked what originally drew them to OpenBSD. In the case of David Gwynne, it was the simplicity of the system:
"The things I’ve always really liked about OpenBSD are its hardware support, its installer, and its documentation. Our hardware support is awesome not because we have drivers for everything out there, but because if we do have a driver it Just Works™. We also make a lot of effort to make it easy to use. For example, we have one tool to configure all our network interfaces called ifconfig, and it does basically everything there is to do with configuring an interface, no matter if it’s wireless, Ethernet, or virtualized. I don’t understand why I need to use several tools on other operating systems to configure an interface."
This simplicity of interface ties in well with one of the new features of 3.9: the sensors framework. The idea behind this is to unify the interface to motherboard sensors. Marco Peereboom explained the advantage of this:
"I am very excited about server management. OpenBSD now supports three major management interfaces in a single uniform way. All other OSes, including Windows, have vendor-specific methods to display, manipulate, and report sensor data. This is a giant mess across the board, however; OpenBSD has started to add some sanity to it. All sensors drivers provide the sensor readings via the sysctl(3) interface which in turn is interpreted and acted upon via the sensorsd(8) framework. It’s extremely simple and straightforward."
David summarizes the advantages, saying:
"It doesn’t matter what type of temperature sensor you have, or where it’s located. It all appears under sysctl hw.sensors in OpenBSD. On other platforms, you have different tools to query different classes of sensors, which really gets in the way of quickly finding out if your computer is too hot or not."
One new feature that caught my eye was support for Apple hardware with the PowerPC G5 series of CPUs. This is noteworthy because OpenBSD has gained this support before NetBSD, the BSD most noted for its portability.
As always, the driver support has improved since the last release. In particular, the NVIDIA Ethernet adaptor on nForce motherboards is now supported, in spite of a lack of documentation from the manufacturer. David points out the potential problem with this:
"Actually, I’m not sure if having the NVIDIA Ethernet driver is a good thing. We used to be able to claim that buying hardware from a vendor who wouldn’t provide documentation was bad for the user because their hardware wouldn’t work. Now their hardware works, but the vendor still hasn’t given us documentation. There’s no incentive there for the user to stop buying that vendor’s hardware and hurt them for their stupid policy with documentation."
He then adds:
"It’s still very cool that we support it with a real driver, though."
Money and a Free OS
Developing OpenBSD costs money. As well as the core OS project, there are several other sub-projects. The most well-known of these is OpenSSH, the most popular SSH implementation, which is used by most UNIX and UNIX-like systems, including most Linux distributions.
The OpenBSD packet filter, pf, is now used by the rest of the BSD family. It is very flexible and easy to configure, and supports load balancing and Quality of Service with a simple to understand syntax. Several ISPs use OpenBSD boxes for routing due to this framework. Next to pf are OpenBGPD and OpenOSPFD, which allow easy cross-network routing.
Recently, work has begun on a BSD-licensed CVS implementation that incorporates the same degree of security as the rest of OpenBSD.
The OpenBSD project costs about $100,000 a year to run. Because the project receives little or no corporate sponsorship, its revenue comes almost exclusively from CD sales and donations. The project does not release full CD downloads onto the Internet, just a basic boot CD that allows network installs, the idea being to encourage users to purchase the official CDs. Unfortunately for the project, it is now very easy to perform network installs, and the vast majority of users do this instead of purchasing the CDs.
Marco Peereboom has been active in drawing public attention to the OpenBSD funding situation recently. He writes:
"Costs are involved with these operations that need to be covered. There are direct costs just to keep the operation going like heating/cooling, power bill, hardware replacements, T1 networking, etc. And there are indirect costs mostly incurred during hackathons. These hackathons are multi-day events where developers from all over the world come together and focus on specific areas of the code. The reason why this is so productive is two-fold:
- People with day-jobs are not "wasting" time and energy at work and get to focus on whatever it is that needs to be done.
- People with widely different skill sets and areas of expertise are readily available to ask questions. Email and ICB (an IRC look-alike) are effective communication mechanisms, but nothing beats good old face-to-face communication when it comes to complex scenarios.
The hackathons are usually where the true innovation comes from and are therefore deemed critical for OpenBSD’s and OpenSSH’s future."
Why Should I Care?
If you run OpenBSD, then you receive the benefit of an easy-to-use UNIX system with a professional level of code quality and security review. Even if you don’t use OpenBSD directly, there are several ways in which you can benefit from the project’s continued existence. Marco explains some of these:
"Lots of people do not realize the impact of OpenBSD on the general open source community. OpenBSD is a very harsh environment for applications due to its stringent security features. The net result of this is that applications fail in OpenBSD while they ’run’ in other OSes. Whenever possible, OpenBSD folks will report and fix this to the upstream developers. The reason why security features in other OSes fail is because they end up disabled; as an example, disabling SELinux is trivial and users who need to run a broken app will do so in favor of getting the application actually fixed."
One example of this is the X.org X server. This is standard on most Linux distributions and several commercial UNIXes. A recent release of OpenBSD included a new malloc implementation, which was more aggressive in catching write-after-free errors. This caused the X server to consistently crash on startup. The bug was eventually traced back to some code that was more than 10 years old. This had been producing intermittent crashes on other platforms throughout the years, but had not been fixed because it was very hard to consistently reproduce. Marco points out the downside of this:
"A sad statistic in all this is that we have been accused of all kinds of ugly things; the often mentioned reason is, ’Well, it works on Linux!’ These folks completely ignore the fact that the app is simply broken and needs patching."
This philosophy is part of the security mentality of the OpenBSD team. The members believe that if something is broken, then it should break, ideally in an easy-to-fix way. A program that continues operating in a way that is not what the developer intended is a potential security risk.
Even if you somehow manage to avoid using any third-party software that has benefited from OpenBSD-supplied bug fixes, the odds are that you have used OpenSSH. To quote Marco:
"OpenSSH is, bar none, the most used open source application on the planet. People don’t even realize how much they rely on this magnificent piece of code. It is deployed in every free and non-free UNIX, and it runs on virtually all routers, switches, and other networking products."
Critics have suggested that OpenSSH and OpenBSD should be funded separately. This is not really a plausible option. OpenSSH is developed by OpenBSD developers, and is an integral part of the OpenBSD user land. It is so well respected because it receives the same level of security review as the rest of the system and testing in the harsh environment of OpenBSD.
The Future of OpenBSD
The OpenBSD project is continuing to move forward, in its traditional incremental fashion. The OpenBSD release schedule calls for a release every six months and so there are usually few user-visible changes between releases.
One area that has undergone a lot of work recently is ACPI support. Producing a complete and correct ACPI implementation is a significant amount of work, and unfortunately isn’t all that is required. A great many ACPI "compliant" BIOSes have significant bugs that need to be worked around. Future releases are expected to contain gradual improvements to this, making OpenBSD an even better choice for laptop deployment.
In 2004, the first OpenBSD release to support SMP was created. The original BSD kernel was designed to run on PDP-11s, and later VAXen, both of which were single-processor designs. As such, the kernel is not designed for concurrency. The simplest way of supporting multiprocessing systems is to put a Big Giant Lock around the entire kernel, ensuring that only one userspace program is allowed to be executing a system call at a time. This was the strategy adopted by older FreeBSD releases, and was the initial step for OpenBSD.
The Big Giant Lock strategy is guaranteed to be correct, because the kernel is still effectively behaving as a single process, and it allows multiple userspace programs to be run concurrently. Unfortunately, it does not scale well. Most programs spend around 10 percent of their time executing system calls. This means that with 11 concurrent processes, you are guaranteed to have at least one waiting for another to complete a system call. Owing to the birthday paradox, this probability doesn’t drop to 50 percent until you have a much smaller number of tasks, meaning that you need significantly more active tasks than processors for this scheme to provide much performance benefit.
The next step, therefore, is to gradually move subsystems out from this lock into a finer-grained system. The correct strategy for doing this is the matter for some debate. In the FreeBSD world, a disagreement over the best strategy led to the forming of DragonFly BSD, which is adopting a microkernel-esque approach with concurrent components communicating over a simple message-passing interface, while the main FreeBSD branch continues to use more traditional threads and locks method. Whatever approach is used, it is very easy to introduce potentially exploitable bugs caused by unforeseen interactions between concurrent components. For this reason, the OpenBSD team is being very careful and methodical about the addition of greater degrees of parallelism into its kernel.
OpenBSD, while perfectly usable on desktop and laptop systems, is traditionally seen as a server operating system, and a lot of development is focused on supporting this use. As such, one significant focus for both current and future development work is better support for RAID hardware. OpenBSD already boasts the only fully open implementations of drivers for a range of RAID hardware, as well as a uniform RAID management interface, and this support is growing.
While the exact future course of OpenBSD depends on the whims of the developers, it is certain that it will continue to provide good clean code with a simple and well documented user interface.
OpenBSD is not the flashiest of operating systems, and not one that gets a lot of attention. My impression of OpenBSD is that it does for the command line what Apple does for the GUI—things Just Work, and work as you would expect them to. Most of the great features of OpenBSD are only apparent when using a system which lacks them.