I think your economic analysis is critically flawed.
(a) What's the source resource in this system? If you really feel that there aren't enough webmasters out there (counting both the affiliate doorway spammers and legitimate personal or organizational sites), and that the global economy needs to shift -- and WILL shift, given opportunity -- to make webmastering more efficient by harnessing underutilized reviewers, then your solution addresses your problem.
If the problem with the ODP is, "not enough reviewers, too many importunate webmasters", then it's obvious that what you'd need to do instead would be make REVIEWING more efficient, at such a high cost to webmasters that the vast majority would be driven out of business.
Obviously, from your proposal, reviewing could be made more efficient by such things as omitting the zero-economic-value feed back for 99.99% of all sites (any reviewer who HAS reviewed ten thousand sites will confirm this figure!); or by replacing the redundancy and delay and ancilliary overhead of compulsory voting with a far-more-efficient single-reviewer process with additional reviews only when needed (needed, that is, by some disinterested party, of course!).
And you'd be back re-inventing the ODP.
There's no reason to agglomerate a website review service for webmasters with a directory for surfers. It reminds me of the old Saturday Night Live advertisements for things like pine-scented carburetor sprays...
There is very likely a good market for website review services, and you could probably create a very interesting website containing links and frank, practical commentary. It would be a lot of work -- billing $10.00 an hour for service worth doing at all would be extremely difficult.
You'd have to be very good at it, or find a way to hire cheap, good labor -- in other words, be a very good manager. (I won't compete with you on either count! The ODP couldn't compete with you--it pays in ideals, whereas what you're describing is strictly a mercenary service -- no social justification other than whatever economic value your customers perceive.)
But ... that's what you need for a real business (as opposed to a fly-by-night scam): something that other people find (a) very difficult to do, and (b) obviously economically valuable. Well, you pass (a); I don't know about (b).
Which brings up the question: how much would webmasters pay for a frank public review of their website? How many websites have you reviewed? Of them, what percentage of those could you have given feedback of obvious economic value?