InformIT: Mac OS X Unleashed

Table of Contents

Darwin

In the standard OS X architecture model, Darwin is represented by a single layer. In reality, Darwin itself is composed of two layers: the Mach kernel and the BSD subsystem. This distinction is important to Mac users because this represents the first time that the consumer Mac OS has been kernel based.

Mach Kernel

A kernel is a small piece of controlling code that abstracts the hardware of a computer from the software that runs on it. The kernel serves as a gatekeeper for all other processes and programs. In a kernel-based system, only the kernel can directly access hardware (I/O systems, memory, and peripherals). By only allowing a single piece of software to perform critical activities, individual applications can no longer crash or corrupt the system. As any user of earlier Mac OS revisions knows, this is not how the Macintosh has worked in the past.

Darwin uses the Mach v.3 kernel, which is highly optimized and stress tested. By ensuring that the base kernel is as stable as possible, stability cascades up to the rest of the OS X architecture. If a program crashes in one of the layers above the Darwin core, the system remains operational. In the traditional Mac operating system, any program can directly access system memory if it so desires. Creating an application that crashes Mac OS 8 or 9 is trivial. This is not the case with Mac OS X.

In addition to stability, the Darwin kernel also provides some unexpected benefits to the rest of the operating system. By implementing new technologies at a kernel level, the benefits are made available to the rest of the system without the need for updating end-user applications. Here are just a few of the technologies that the kernel makes available to the rest of the Mac OS X operating system transparently:

Protected Memory— In the traditional Mac OS, each application could request memory from the system and it was up to that application to not write to memory outside of the available space. The system, however, couldn't prevent this from happening. In Mac OS X, each application runs in its own memory space. It may not access the memory space of the system or of other running applications (unless running with advanced permissions). The use of protected memory also means that one can use the "force quit" option of the operating system without the need to reboot to regain system stability.
Virtual Memory— The mach kernel handles all memory access and can dynamically allocate virtual memory as needed. This replaces the user-defined swap file of earlier Mac operating systems. Memory management is now handled automatically with no interaction needed.
Pre-emptive Multitasking— Under Mac OS X, applications cannot lock up your computer by taking up all the computer's processing time. Long-time Mac users will appreciate this feature because we've all waited minutes for our systems to come back when Netscape hits a complex page. In Mac OS X, when even a single application is chugging away or has become unresponsive, you can switch to another program and keep on working.
Symmetric Multiprocessing— SMP turns the Macintosh into a highly scalable computing platform. Several models of the Macintosh have included multiple processors, but only applications that have been explicitly programmed for the Mac OS multiprocessing API could take advantage of them. In Mac OS X, SMP is handled at the kernel level, allowing basic tasks at all levels of the operating system to take advantage of multiple processors.
Network Kernel Extensions— The advanced networking model of the Darwin kernel allows protocol stacks to be loaded and unloaded dynamically, as well as real-time monitoring and modification of network traffic. This allows complex network applications such as firewalls to be built easily under Mac OS X.
Multiple File System Support— The default file system under earlier Mac OS versions and Mac OS X is HFS+. Additional file systems can be added as modules to the kernel. After they're added, these file systems are available to all applications running on the system, regardless of whether the program understands the foreign file structure.

Although some of these features might seem complex and not something that you would want to deal with, the good news is that you don't have to. The Darwin kernel abstracts all the technical details from the end user. While developers can create new modules that operate at the kernel level, end users need do nothing more than sit back and reap the benefits.

There is one potential benefit of a kernel-based system that has not yet been exploited by Apple, but it might very well change the face of the Macintosh forever.

Since its inception, the Macintosh has been tied to the Motorola processor. First, the Motorola 68000-series CPU provided the computing power through the early 1990s. In 1994, Apple moved to the PowerPC platform, which is still in use today. The PowerPC platform, which uses a RISC architecture, offers some speed advantages over the traditional CISC (Pentium) design. Unfortunately, it also means that Apple is dependent on IBM and Motorola (the PowerPC development partners) for the advancement of the Macintosh platform.

In the early 2000s, the megahertz wars started to make a dent in Apple's apparent speed advantage. Intel and AMD pushed their processors above 1GHz, while the PowerPC finally reached 733MHz in early 2001. Although it's faster for some tasks, the PowerPC cannot keep up with AMD's latest offerings overall.

All of this might soon change if Apple decides to exploit the kernel-based nature of its operating system by porting it to other processing platforms. The Darwin kernel has already been ported to the Intel platform. With minor work, it's very possible that Mac OS X could be deployed on Intel or AMD-based systems. In fact, up until the very final release of Mac OS X Server, the Rhapsody Developer releases also ran on the Intel platform! Even though there is no official word from Cupertino, many speculate that Apple is secretly maintaining an Intel-based version of Mac OS X somewhere in its development labs.

The BSD Subsystem

Above the Darwin kernel lays the BSD (Berkeley Software Distribution) subsystem. This portion of the Mac OS X operating system was designed never to be visible to end users unless they wanted to use it. We, the authors of Mac OS X Unleashed, hope that with the help of this book, you will be able to fully exploit the BSD portion of Mac OS X to accomplish tasks that were never before possible on the Mac platform.

So, what is BSD? The simple answer is that BSD is a collection of software that makes up a Unix-like operating system. Unfortunately, this leads to the question, what is Unix? This is a little bit more difficult to answer.

Unix is a multi-tasking multi-user operating system developed at Bell Labs during the 1970s. It was created to be a stable and powerful development platform for programmers. Today, Unix is largely regarded as a highly cryptic operating system that sends today's coddled point-and-click Windows NT administrators running in fear of actually having to think. Unix is composed of hundreds of different programs that work together to provide access to files and other standard operating system functions.

There are two primary distributions of Unix. The first is the Berkeley Software Distribution (BSD), which (obviously) was developed at Berkeley. The second distribution is System V, which was developed as a commercial Unix by AT&T. Although many of the functional differences are minor, the philosophies behind the two distributions are much different. BSD is preferred by academic institutions and hobbyists, whereas System V is used in commercial Unix distributions.

Many people have asked why Apple didn't choose to base its new OS on the very popular Linux operating system. What those people fail to understand is that Linux is actually a kernel, not a complete operating system. The Linux kernel, although interesting, is extremely unstable in terms of development. Linux users grow accustomed to updating their systems several times a year. This is fine for power users, but it isn't appropriate for production environments or home users. The Mach kernel is time-tested and provides a stable operating environment.

Linux users, however, will find themselves at home on Mac OS X, because the software that runs on top of the kernel is largely the same as popular Linux distributions. The same programs and scripts that run on Linux can be easily configured to operate on Mac OS X.

Under BSD, tasks such as listing the files in a directory (folder) work a bit differently than in the standard Mac OS. Let's take a look at a few examples of exactly what the BSD subsystem looks like. This marks the first time that the Mac OS has had a true command line, so brace yourself!

First, let's take a look at the actions needed to view a directory listing of files. Under traditional Mac operating systems, you simply double-click a folder, switch to List view, and look at your files, as seen in Figure 1.2.

Figure 1.2 Viewing a file list in Mac OS is as easy as double-clicking a folder icon.

Under the BSD subsystem, you can use the ls command to generate a file listing. For example, the following is a list of the top level of a Mac OS X hard drive:

[localhost:/] jray% ls -al
ls: Music: Permission denied
total 18216
drwxr-xr-x  36 root  admin     1180 Apr  1 10:52 .
drwxr-xr-x  36 root  admin     1180 Apr  1 10:52 ..
-rw-rw-rw-   1 root  admin     8220 Mar 31 16:28 .DS_Store
d-wx-wx-wx   3 root  admin      264 Mar 30 18:45 .Trashes
-r--r--r--   1 root  wheel      142 Feb 25 03:05 .hidden
dr--r--r--   2 root  wheel       96 Apr  1 10:51 .vol
-rwxrwxrwx   1 root  wheel   688128 Mar 31 10:16 AppleShare PDS
drwxrwxr-x  26 root  admin      840 Mar 30 18:45 Applications
drwxrwxrwx  12 root  wheel      364 Mar 27 22:34 Applications (Mac OS 9)
-rw-r--r--   1 root  admin   688128 Mar 31 00:09 Desktop DB
-rw-r--r--   1 root  admin  3170274 Mar 31 14:50 Desktop DF
drwxrwxrwx   9 root  staff      264 Mar 30 18:30 Desktop Folder
drwxrwxr-x  13 root  admin      398 Mar 10 00:05 Developer
-rw-r--r--   1 jray  admin        0 Mar 26 22:55 Icon?
drwxrwxr-x  21 root  admin      670 Mar 24 00:35 Library
drwxrwxrwx  19 root  wheel      602 Mar 20 23:36 Mac OS 9
drwxr-xr-x   7 root  wheel      264 Mar 24 00:35 Network
drwxr-xr-x   3 root  wheel       58 Mar 23 10:02 System
drwxrwxrwx   3 root  wheel      264 Mar 31 15:06 Temporary Items
drwxrwxrwx   4 root  wheel      264 Mar  9 21:03 TheVolumeSettingsFolder
drwxrwxrwx   5 root  wheel      264 Mar 31 10:16 Trash
drwxr-xr-x   5 root  wheel      126 Mar 26 20:35 Users
drwxrwxrwt   2 root  wheel      264 Mar 31 15:49 Volumes
drwxr-xr-x  33 root  wheel     1078 Mar  1 21:03 bin
lrwxr-xr-x   1 root  admin       13 Apr  1 10:52 cores -> private/cores
dr-xr-xr-x   2 root  wheel      512 Apr  1 10:51 dev
lrwxr-xr-x   1 root  admin       11 Apr  1 10:52 etc -> private/etc
lrwxr-xr-x   1 root  admin        9 Apr  1 10:52 mach -> /mach.sym
-r--r--r--   1 root  admin   652056 Apr  1 10:52 mach.sym
-rw-r--r--   1 root  wheel  4039448 Mar  1 09:58 mach_kernel
drwxr-xr-x   7 root  wheel      264 Apr  1 10:52 private
drwxr-xr-x  56 root  wheel     1860 Mar  1 21:01 sbin
lrwxr-xr-x   1 root  admin       11 Apr  1 10:52 tmp -> private/tmp
drwxr-xr-x  11 root  wheel      330 Mar 26 14:31 usr
lrwxr-xr-x   1 root  admin       11 Apr  1 10:52 var -> private/var

Instead of a graphical representation of the files, their sizes, and their modification dates, information is presented in a text format. In this particular example, a variety of files and directories are foreign to the Macintosh community: Desktop DB, mach.kernel, usr, var, etc, and so on. These elements are hidden from the Mac OS X end user, and do not need to be modified unless you are interested in tweaking the system from the BSD level. Information for performing file listings and other common tasks will be introduced in Chapter 12, "Introducing BSD."

Caution

Unlike previous versions of the Mac OS, Mac OS X does not store all the operating system–critical files in a single system folder. Although (as seen in the listing) there is a System folder that does contain important portions of the operating system, it does not contain the BSD subsystem. The BSD portion of the operating system is hidden in the private, sbin, var, usr, tmp, bin, and dev directories. Although they are not normally visible, if the names or locations of these directories are changed, Mac OS X is likely to become inoperable.

Let's take a look at another common task on Mac OS—viewing the applications that are currently running on the system. Under versions of Mac OS earlier than X, the running programs are shown in the Application Switcher menu or tear-off palette, displayed in Figure 1.3.

Figure 1.3 Mac users are used to seeing lists of running tasks represented graphically within the Application Switcher.

In Mac OS X, the active applications are moved from the task manager menu/palette to the Dock, which offers another visual representation of current processes. To view the active processes under the BSD subsystem, however, one would use ps to list the active processes. For example, I can view all the processes on my Mac OS X computer by typing ps axg at a command-line prompt:

[localhost:~] jray% ps axg
  PID  TT  STAT      TIME COMMAND
    1  ??  SLs    0:00.03 /sbin/init
    2  ??  SL     0:01.10 /sbin/mach_init
   38  ??  Ss     0:01.58 kextd
   56  ??  Ss     0:04.14 /System/Library/Frameworks/ApplicationServices.framew
   58  ??  Ss     1:43.61 /System/Library/CoreServices/WindowServer
   60  ??  Ss     0:01.96 update
   63  ??  Ss     0:00.00 dynamic_pager -H 40000000 -L 160000000 -S 80000000 -F
   85  ??  Ss     0:00.41 autodiskmount -v -a
  104  ??  Ss     0:01.39 configd
  113  ??  Ss     0:00.03 ipconfigd
  149  ??  Ss     0:00.14 syslogd
  160  ??  S<s    0:00.67 /usr/local/sharity/sbin/sharityd -f /usr/local/sharit
  188  ??  Ss     0:00.02 portmap
  191  ??  Ss     0:00.04 nibindd
  192  ??  S      0:00.22 netinfod local (master)
  199  ??  Ss     0:01.15 lookupd
  208  ??  S<s    0:01.33 ntpd -f /var/run/ntp.drift -p /var/run/ntpd.pid
  213  ??  Ss     0:00.02 DesktopDB
  218  ??  Ss     0:00.00 inetd
  228  ??  S      0:00.00 nfsiod -n 4
  229  ??  S      0:00.00 nfsiod -n 4
  230  ??  S      0:00.00 nfsiod -n 4
  231  ??  S      0:00.00 nfsiod -n 4
  240  ??  Ss     0:00.00 automount -m /Network/Servers -fstab
  252  ??  Ss     0:00.29 /System/Library/CoreServices/SecurityServer
  276  ??  Ss     0:00.49 /usr/sbin/sshd
  279  ??  Ss     0:00.01 cron
  282  ??  Ss     0:06.68 /System/Library/CoreServices/loginwindow.app/loginwin
  285  ??  S      0:02.67 /System/Library/CoreServices/pbs -psn_0_262145
  286  ??  S      0:03.51 /System/Library/CoreServices/Finder.app/Contents/MacO
  287  ??  S      0:05.09 /System/Library/CoreServices/Dock.app/Contents/MacOS/
  288  ??  S      0:50.01 /Applications/Mail.app/Contents/MacOS/Mail -psn_0_655
  289  ??  S      0:04.33 /System/Library/CoreServices/DocklingServer.app/Conte
  290  ??  S      0:23.27 /Applications/Network Stuff/Fire.app/Contents/MacOS/F
  291  ??  Ss     0:00.56 /System/Library/Services/AppleSpell.service/AppleSpel
  292  ??  S      5:14.81 /Applications/Network Stuff/OmniWeb.app/Contents/MacO
  308  ??  S      0:09.61 /Applications/Utilities/Terminal.app/Contents/MacOS/T
  334  ??  S      0:04.08 /Applications/System Preferences.app/Contents/MacOS/S
  335  ??  Ss     0:00.65 /System/Library/PrivateFrameworks/Admin.framework/Res
  381  ??  Ss     9:40.62 /System/Library/Frameworks/ScreenSaver.framework/Vers
  386  ??  S      0:00.05 /usr/sbin/sshd
  309  p1  Ss+    0:00.25 -tcsh (tcsh)

A close inspection of the process listing shows a few understandable lines that contain Mail.app (the Mac OS X e-mail application), Terminal.app (the application used to access the command line), and a few other goodies; largely, it is filled with seemingly meaningless entries.

These processes, such as nibindd and lookupd, are integral parts of the Mac OS X operating system. Unix and Mac OS X were designed so that multiple small programs would work together to provide robust services system-wide. When using the Mac OS X user interface, they do not appear in the Dock, but they are running nonetheless.

Next, let's take a look at the use of Sherlock to find files on the system. In Mac OS X and earlier, you can simply invoke the Sherlock application by pressing Command+F. To find a file, just type its name and press Enter. Figure 1.4 shows a Sherlock window with a search for filenames that contain the word Java.

Figure 1.4 Sherlock provides a fast way to locate files, but the locate command is faster.

The BSD system has an equivalent feature that is actually faster at locating files than Sherlock. Appropriately enough, the command is locate.

[localhost:~] jray% locate Java
/Trash/Rescued items from Picasso 1/QTJava.zip
/usr/lib/java/libAdminJava.A.dylib
/usr/lib/java/libAdminJava.dylib
/usr/lib/java/libAppKitJava.B.dylib
/usr/lib/java/libAppKitJava.dylib
/usr/lib/java/libFoundationJava.B.dylib
/usr/lib/java/libFoundationJava.dylib
/usr/lib/java/libNIAccessJava.B.dylib
/usr/lib/java/libNIAccessJava.dylib
/usr/lib/java/libNIInterfaceJava.B.dylib
/usr/lib/java/libNIInterfaceJava.dylib
/usr/lib/java/libObjCJava.A.dylib
/usr/lib/java/libObjCJava.dylib
/usr/lib/java/libPreferencesJava.A.dylib
/usr/lib/java/libPreferencesJava.dylib
/usr/lib/java/libPureAppKitJava.B.dylib
/usr/lib/java/libPureAppKitJava.dylib

Within one or two seconds of issuing a locate command, all the files matching the supplied command-line string are printed. In the case of Java, more than 3,000 files were found that include Java in the pathname. The example shown here has been trimmed quite a bit; otherwise, the rest of the chapter would consist of nothing more than a very long file listing.

Finally, there is one very unique benefit to the Unix command-line environment: the ability to use two commands together that have nothing to do with one another. This is accomplished by redirecting the output of one command into the input of another. For example, in the previous example, I stated that there were more than 3,000 entries that matched Java in the pathname. No, I wasn't crazy enough to actually count all the files. Instead, I used the wc command to count the number of lines in the output of the locate command:

[localhost:~] jray% locate Java | wc -l
    3196

The locate command sends its output to wc -l, which counts the number of text lines that are handed off to it. The | (pipe) symbol is used to connect the output of locate to the input of wc.

Using similar techniques, you can create very complex utilities that can connect to remote hosts, download information, and process that information without ever writing a line of code yourself.

The command line provides a very powerful resource for advanced users and adds an entirely new dimension to the Macintosh operating system.

Darwin's Open Source Innards

One of the most intriguing aspects of Darwin is that it is open source software. The Open Source software movement has been the driving force behind such powerhouses as the Apache Web server and the Linux distributions.

Open source software takes a different approach to the development of applications. Rather than confining the programming to an internal team of developers, open source projects can be accessed by anyone. Changes to the application are submitted back into the source code tree and integrated into the final product. This empowers people who are interested in improving the Mac OS to work on the Darwin source code and create a more powerful operating system for everyone.

The Open Source movement enables users to inspect the code that is running on their computer. This is in direct contrast to proprietary systems such as Microsoft Windows, where the user can only assume that the parent company is making the appropriate decisions on how the software should operate. The success of open source software relies on the people, not a particular company.

Apple is the first major computer manufacturer to embrace the Open Source movement by placing several key components of its software library under the Apple Public Source License. Both Darwin and the QuickTime Streaming Server are APSL projects and have been warmly received by the developer community. In fact, Darwin has been ported to the Intel platform and QuickTime Streaming Server is available for Linux and NT—courtesy of open source developers. Projects are underway to bring additional driver support to Darwin, which, in turn, will bring the same support to the commercial version of Mac OS X.

Note

Apple's APSL has not been without criticism from the open source community itself. Most open source software falls under the GNU General Public License (http://www.gnu.org/copyleft/gpl.html). Apple's APSL, however, exerts ownership of any code submitted to the project and does not require that Apple's own modifications fall under the license. At this time, this has not proven to be an issue with any of the projects, but it is still a cause for alarm among some open source proponents. Learn more about Apple's open source projects at http://www.opensource.apple.com.

Interestingly enough, Darwin is a complete operating system in and of itself. Interested users can download the Darwin installation and have a full working operating system without paying a dime for admission. Apple has not placed any of the other Mac OS X components under the APSL, so components such as Carbon, Cocoa, and Quartz remain proprietary software. Regardless, this represents a big step forward for the computer industry and is likely to be repeated industry-wide for many other popular applications.

Darwin is an extremely important component of the total Mac OS X operating system and will be the focus for several chapters of this book. In many respects, Mac OS X is a composite of two operating systems. The BSD system is a completely functional environment in its own right. Mac OS X, however, is known to its users as a lush graphical environment with a cutting-edge interface and advanced Mac-like operation. Together they combine to form a system that is as appropriate in an elementary school classroom as it is in a server room.

QuickTime | Next Section

Previous Section