Another curiosity question....

Robots? Cloaking? Ads? (Oh my?)

[evinrude]

A site I visited for the first time had a link to robots.cnn.com [robots.cnn.com], which would appear to be CNNs regular web page minus all the ads and the netscape toolbar.

Since it starts out robots, I'm assuming is used for things such as offline viewers, or internal scripts. I thought for a bit perhaps search engines, but on a site that can change on a minute to minute basis, I'm not so sure. :)

Having only recently gotten into search engine optimisation, I'm curious what the benefit of a page like this would be. Is this an example of cloaking? Or am I missing some other, more important, point.

Anyway, just curious. :)

[Marcia]

evinrude, take a look at the robots.txt for that - it's interesting.

News is being indexed very rapidly now, so there may be a connection. Have you seen this thread on what FAST is doing?

[webmasterworld.com...]