barbos - 2:02 am on May 3, 2007 (gmt 0) It is not clear though whether slurp will follow the links within class="robots-nocontent".
[ysearchblog.com...] and [help.yahoo.com...]
...webmasters can now mark parts of a page with a 'robots-nocontent' tag which will indicate to our crawler what parts of a page are unrelated to the main content and are only useful for visitors. We won't use the terms contained in these special tagged sections as information for finding the page or for the abstract in the search results. Note: Using a "nocontent" tag to mark explicit sections of content is not considered "cloaking" because all of the content on the page is available to protect the relevance of the results (unlike "cloaking" where we may be served content that is different from what visitors see).
It is not clear though whether slurp will follow the links within class="robots-nocontent".