InformIT

Searching for Substance: Web Browser Olympic Scorecard

Date: Oct 1, 2004

Return to the article

Nigel McFarlane boils down all the competition between browsers into a single score for each browser. Take this article to your local second-hand browser dealer to test his honesty.

A previous version of this column appeared with a piece describing technology politics between Microsoft and the World Wide Web. The echo of that competition has now reached everyday web users in the shape of a new web browser mini-war. That war is fought mostly between Internet Explorer and open source products such as Mozilla and Firefox. Differences between browsers can be hard to spot, though, and the whole thing might well mystify you.

Browser Olympics

With the Olympics and their medal tallies not far behind us, it's worth asking: What should a web browser strive for? A web browser should strive to give the user perfect access to the World Wide Web. Not good access or great access, or this week's access—perfect access. That goal might sound achievable, but it turns out to be something of a technical Holy Grail. It turns out that issues such as big buttons and smart, labor-saving features are far easier to achieve than silent, perfect access. Perfect access is what web surfers need, though. If the TV has drifted off the station, putting it into a nice cabinet won't help. You need a clear picture.

So let's have Browser Olympics.

Competitors

There are four main contestants—with deadline apologies to the other non-qualifiers. The contestants each provide a core set of web engine technologies that support the web. Let's line up our competitors:

All of these engines are used in several end-user browser products. For example, the Mozilla engine is used in the Firefox, Epiphany, Netscape, and Camino browsers—to name just a few.

Scoring

Just as in the real Olympics, we need referees and rules so that the contest is fair. All of the four main contestants or their backers have contributed to the public standards that make up the World Wide Web. These standards make a promise to end users that the World Wide Web is a global commons—a place where everyone can participate. Web standards are therefore a fit and familiar place for our contestants to fight it out. They've all been in tourneys; they've all been jousting on the Net for some time.

Another battlefield for our contestants is security. Security is a bit like the Olympic marathon; anyone can enter—finishing in style is the problem.

For each event, we'll score the contestants between zero (0) and one (1). It's technology, after all. Since technology is pretty complex, we'll allow four possible scores, as shown in the following table.

Score

Meaning

1.0

Performs perfectly.

0.99

Perfect except for a few weird and minor problems that have yet to be chased down.

0.9

A fair way toward perfection, but with a few things yet to achieve.

0

Breaks the contest rules on purpose (not trying, cheating, just plain belligerent—none of that makes for a fair comparison).


Contestants are all allowed to use their shiniest, best versions in our five events:

Event 1: Recognize the Type of Any Web Document

Contestant, please grab a page from the web and tell the user what that page advertises itself to be. This is a simple warm-up event, and a sanity check for users. To perform well in this event, the engine has to meet HTTP and MIME standards. That's not very hard, but it's made harder than it needs to be because web servers can deliver all kinds of junk to web browsers. Still, browsers are within their rights to recognize some types and give up on others. That's in the rules.

There's little to say about most of the contestants, since they dutifully report what the web tells them. Internet Explorer is the complicated exception. The difficulty IE has is that it's trying to be two things:

The desktop has a set of rules for recognizing files. That set of rules is different from the web rules. IE tries to mash together the two rule sets so that the desktop is web-like, and the web is desktop-like. That nearly worked, but then the U.S. Department of Justice intervened. The upshot is that IE's compromise solution sometimes contradicts what the web has to say. The web has remained independent of Microsoft technology and IE provides less-than-perfect access for users.

Event 1 Results

Competitor

Score

Mozilla

1

Opera

1

KHTML

1

Internet Explorer

0.9


Event 2: Recognize the Content of Any Web Document

Contestant, please examine a web document of unknown origin, and tell the user exactly what kind of content is in it. A more difficult event. It turns out that there are lots of subtle variations in web content, and a web surfer wants a browser smart enough to detect all of those variations. You don't want to surf half the web.

Event 2 Results

Competitor

Score

Mozilla

0.99

Opera

0.99

KHTML

0.99

Internet Explorer

0


Why no perfect scores? The problem is that the many kinds of HTML and XHTML web pages can be divided into two groups: ones that are made well, and ones that are made badly.

It's not in the rules, but all browsers try to provide some support for badly made web pages. That's an important feature, because it means that school kids, the disabled, the uneducated, and harried managers can all be heard across the web. There should be extra points here, but the Olympics are a tough competition. Instead, we have points for correct document sniffing of properly made web pages. Most engines sniff documents pretty well, but it's a black art; it's doubtful that any browser will ever be perfect. IE is not as functional as the other browsers. It's a bit older in some ways. IE's unlucky score of zero is due to some very strange rules for handling XML content. The whole world is gradually turning to XML, and for IE to drop the ball on that item is really a big problem for everyone. We'd love to give IE a 0.9, but the way it works it just won't perform to that level.

Event 3: Understand the Content of the Web Document

Contestant, please take this web document and correctly break it down into pieces. We want to know whether you can cope with fancy web pages.

This event is about handling tags, scripts, and other web content, otherwise called parsing. Parsing can be a messy treasure hunt and a chaotic balancing act of special-case processing. Because all the various standards are extensible, any contestant can declare special extra features. Contestants have to meet the standards (gaining points), but they also have to cope with each others' special features (no points there, but also no choice).

Event 3 Results

Competitor

Score

Mozilla

0.99

Opera

0.9

KHTML

0.9

Internet Explorer

0.9


More than in any other event, this one has a degree of subjective judging. As in the Olympic 20-kilometer walk, a few tiny missteps will take you into oblivion. Opera and KHTML both clearly document a few unaddressed standard features, therefore earning 0.9, and both respect Mozilla as the best contestant to date, thus 0.99. Internet Explorer has many unusual features, and although eyebrows might rise at its habit of putting XML data islands inside SGML-based HTML, or for going beyond the special rules allowing XHTML parsing as HTML, it also gathers 0.9 here. Parsing is too gray an area to be handing out brickbats willy-nilly.

Event 4: Display the Web Document

Contestant, please take the content of this web document and paint it onscreen so that the reader can read it. We want to see how faithfully and precisely you do that. In particular, we don't want you changing some author's web content to suit yourself. If we wanted saccharine censorship, we could watch TV. If browsers start fiddling with publishers' content for no good reason, we're all on a slippery slope to somewhere unpleasant. None of that, thanks. Don't blow it, just show it.

This event is called rendering, and although the contestants all behave in a stylish manner, it's still a technical competition, just like Olympic gymnastics. The performance of each browser again depends on standards support.

Event 4 Results

Competitor

Score

Mozilla

0.9

Opera

0.99

KHTML

0.9

Internet Explorer

0 or 1


In this area Opera shines, meeting the CSS 2.1 standard completely. The Mozilla engine's "outline" support is still coming, and that's enough to let it slip back a little. KHTML's CSS support is far less finished than Mozilla's, but there's nothing between 0.9 and 0, so 0.9 it is.

Because IE has historically been popular, many web pages are created explicitly to look good in IE. It's a political business to argue where the true web standards are. Are they the standards made in the castle that is the World Wide Web Consortium (W3C)? Or are they the reality on the ground—whatever IE does is the standard? One thing is certain: IE and the W3C's rendering standards are pretty different, especially with regard to the latest recommendation, CSS 2.1. If you're a purist, you might say that the other browsers seem to manage to follow standards, so 0 for IE. If you're a pragmatist, you'll say that until IE improves (or its market share falls a lot), you'll support both approaches, so 1 point for IE.

Event 5: Keep Users Safe from Villains on the Web

Contestant, please demonstrate that you provide a secure user experience for the web surfer. Don't let access to the web wreck the user's computer or deliver private information to intruders.

Event 1 Results

Competitor

Score

Mozilla

0.99

Opera

0.99

KHTML

0.99

Internet Explorer

0.99


This event is very difficult to judge. The problem is that if a security issue becomes evident, most browser vendors are very quick to fix their engines. So all the engines appear secure nearly all the time—even if they're not. Furthermore, if no security problems were ever identified with an engine, you'd have to start wondering whether anyone has even bothered to check it. That's an uneasy feeling. Popularity is also a problem—if you're popular, then older, more insecure versions of your engine are likely to be hanging around everywhere, still full of holes. That's bad for your reputation. Because all the engines' latest versions are up to date, perhaps all should score 1. But there's an odd problem or two with the web that engines can't do much about, such as social engineering attacks (jargon for web pages whose content contains old-fashioned lies). Browser engines can't fix that.

Olympic scoring aside, CERT, the U.S. federal security body, records past security problems with our four contestants, as shown in the following table.

Competitor

Vulnerability Notes Recorded by CERT (Search)

Mozilla

4

Opera

1

KHTML (plus Safari)

3

Internet Explorer

60


Totaling the Scores

With the five events completed, the competition totals are in. Out of a possible 5.0, each contestant scored as shown in the following table.

Competitor

Score

Mozilla

4.87 (97%)

Opera

4.87 (97%)

KHTML

4.78 (96%)

Internet Explorer

2.79 (56%) or 3.79 (76%)


The tie between Mozilla and Opera is no doubt contentious. Such a tie is good for consumers, though. Even if IE gains points for setting de facto display standards instead of using public standards, it still lags behind the other browsers. It's pretty clear from these scores why the confidence of the other browser engine makers is sky high at the moment. IE is being left behind, and anyone who has done this kind of analysis knows it. No wonder there's a browser war.

800 East 96th Street, Indianapolis, Indiana 46240