The W3C decided it would not be developing HTML any further, to focus exclusively on XHTML2, and so for the next 9 years the W3C ignored the current web.
Yet still, during all these 9 years, a good deal of the web ignored "the current web", as XHTML1 steadely gained ground over tag-soup HTML, even despite the fact the most used browser (IE) refuses to try to handle it when served properly (which makes no sense, since it is perfectly capable of rendering it, which can be proved by the fact that a quine-ish XSLT gets it rendering). Anyway, I wouldn't take too seriously a browser that has taken over 10 years before decently supporting CSS2.
It wasn't just browser makers who thought XHTML2 was impractical, overly-complex and mostly pointless. Even Zeldman, the high priest of XHTML-advocacy, said he couldn't see any benefits.
I have no idea who that Zeldman is, and even less why is "the high priest of XHTML-advocacy"; but someone who "couldn't see any benefits" on XHTML2 is either blind, hasn't looked at XHTML2 at all, or is simply lying. A few examples from the top of my mind would be support for extensible semantics, decent fallback for <img>, or a cleaner and clearer content model. Whether the benefits are worth the drawbacks and/or the costs could be worth discussing, but that's quite a different thing than saying there are no benefits.
Some folks eventually got so fed up with this situation they decided that if the W3C was not going to develop standards for the current web they would, and setup the WHATWG.
Curiously, these folks are three of the four major browser vendors (five, if you count Google and their recent Chrome browser): Apple, the Mozilla Foundation, and Opera Software. And, as fed up as they were of being ignored by the W3C, they are ignoring the same way every feedback that doesn't come in the form "this browser has implemented this" or as a praise. What's still surprising is that the spec is being written in English and not in C. Have you noticed that it defines more algorythms than markup components (elements + attributes)? And it's supposed to be the spec for the new HyperText Markup Language. That's, at least, curious.
If you believe XHTML2 is the way forward
I don't. It has serious flaws that would need to be fixed for it to be really useful. For a time I thought that HTML5 would provide those fixes, but it just tosses away everything given by XHTML2, regardless of if it's good or bad, and furtherly bloats the already bloated markup HTML4 left us, up to the point of rescuing <i>, <b>, and <small>, among other 3.2-ish presentational aberrations.
then HTML5 provides a stopgap solution to the decade-long stasis that the web has suffered under the leadership of the W3C, until XHTML2 takes over.
There are two separate points here: first, you are asumming HTML5 will come sooner than XHTML2, but most of XHTML2 is already stable, while the WHATWG doesn't expect HTML5 to be stable sooner than 2012. On the second point, about the decade-long stasis of the web, I must say that the role of the W3C has been completely irrelevant to it. Even assuming your idea that W3C's choices were all wrong and ill-fated, and whatever you want, how much matters how long does the W3C to define the next standard if the most used browser (IE) doesn't yet support a 1998 standard (CSS2)? (Ok, the IE8 betas finally seem to support it, but they are still betas.)
OTOH if, like me, you're unconvinced by the case for XHTML on the web, HTML5 provides an alternative path for future development
Indeed. I, like you, am unconvinced about XHTML: there are some things that I like, and many I don't. And effectively, HTML5 provides an alternative, but it's not that better than X2. It's like choosing between jumping into a huge abbyss or into a lava river, only that you don't really get to choose: browsers will come and push you into their own choice.
[edited by: Herenvardo at 12:17 am (utc) on Nov. 3, 2008]