Search engines should embrace cloaking as a way to compensate for the difficulties search engines have in interpreting content appeal.
If we look at search engine spiders as a target audience on our websites, we would find that they are a distinct target audience that behaves very differently from a human.
Differences between a human and a search engine as a target audience
1. Search engines have a hard time interpreting the graphics and general look and feel of a site. For a typical user a great picture is worth a thousand words.
2. Search engines have a hard time interpreting the appeal of a time sensitive offer, such as what you find at an online auction. Would you think to ever see a search result for the keyword "exciting" be an extreamly popular eBay auction listing with 2 minutes to go?
3. Search engines have a hard time interpreting a time sensitive offer. For example, you won't see a "Only 3 left at $19.95 - buy today and it will arrive by Christmas" as a top search result for a correlating specific product keyword.
4. Search engines have a hard time interpreting an emotional appeal. Let's say that another earthquake strikes Haiti, then the Red Cross showing up high in the results for a search of "Haiti earthquake" might be an entirely relevant search result.
Content Distribution Method Appeal
1. Search engines don't share tips with each other on which sites are the most authoritative or relevant, but humans give and share this advice freely.
2. Search engines have a hard time determining the relevancy and authority of an online ad campaign. You won't see a search engine crawl a popular PPC ad and then put the landing page of that ad high in the organic search results.
3. Search engines don't sample offline marketing efforts to determine the relevancy and authority of an online result. Will there be a day when telemarketing, pamphleting, billboards or other offline marketing methods boost your search engine rankings?
4. Search engines don't consider your browser device type as a way to filter search results. But device types can say something about the user personality and intent. Would a search engine ever deliver different organic results for two people in San Francisco searching for a place to eat lunch, one searching on a Blackberry and the other on dial-up?
5. Search engines don't sample email communication in determining the value or placement of an organic result. Many things get forwarded on that would give relevant and authoritative insight on the value of a specific piece of content which would improve a search result - a search result that probably deals with cats and political opinions. :)
6. Search engines like sitemaps, but a sitemap isn't very well received by humans who prefer an enticing display of link options to a lifeless list of links.
Until search engines can better interpret relevant and authoritative content, they should trust websites to send the user/spider to the most relevant page.
Besides, it is probably an easier problem for a search engine to identify an abusive website than to try to match a single search result up with a single user.