The WHAT Working Group?
The last official HTML specification was released in 1999. Microsoft, having effectively won the browser war at that point, stopped updating Internet Explorer, and there was a temporary lull. The W3C was busy with XHTML 2, but it looked increasingly unusable for real-world web design. In 2004, Apple, Mozilla, and Opera formed the Web Hypertext Application Technology Working Group (WHATWG) to develop a more incremental improvement of HTML, which they dubbed HTML 5.
HTML 5 is developed in a more pragmatic way. At any given time, there is a single editor who has final say over what goes into the spec. There is also a committee but, unlike W3C committees that have control over the specification, the WHATWG committee only has the power to replace the editor.
In the late 1990s, Microsoft got into trouble for implementing a draft CSS specification. Developers started using this for their sites, and then the final version was released. Unfortunately, the final version was incompatible. Microsoft had to either support the new one and break a load of existing sites, or support the old one and not comply with standards. In the end, they tried to compromise and do both, which is part of the reason web pages can display differently in IE in different contexts; it will attempt to work out which version of the spec the author wanted and pick the right one.
To avoid this kind of problem with HTML 5, each section of the specification has an independent maturity level. Some parts are mature and well-implemented by browsers, while others are still experimental. Anyone using the experimental parts in their sites should do so on the understanding that they may need to update them as the spec evolves: None of the browser manufacturers makes any commitment to supporting older drafts.
For any part of the specification to be advanced, there must be at least two working, publicly-available, interoperable, implementations. Features can't even be proposed until they have at least one implementation. This helps to avoid the “wouldn't it be nice if…” sort of reasoning that plagued XHTML 2.
As of 2007, the W3C has adopted HTML 5, and it is likely that the final version of the standard will be out around 2012. It's worth noting that HTML 5 doesn't abandon XHTML. The specification separates the set of tags that should be supported and their abstract meanings from how they are represented. There are HTML bindings and XML bindings allowing you to use these tags in either SGML-like or XML-like documents. If you use the former, you can write the traditional sort of tag soup, with no requirement to close most tags. If you use the latter, you can mix other XML formats into your document.
Although it began as an everyone-except-Microsoft group, IE 8 has partial support for HTML 5, and Microsoft has announced HTML5 support as a feature of the upcoming IE9. That said, HTML 5 is designed in such a way that it can be supported in older browsers via scripts and plugins, so this support is not essential.