The second result is their robots.txt file - well, it made me laugh.
[edited by: Brett_Tabke at 1:36 pm (utc) on Sep. 9, 2005] [edit reason] lets not do specifics... [/edit]
Look at #4: [google.com...]
txt = no formatting xml = so much foramtting you cant read it
Both are silly results imo
They don't have to have high rank, and if there is no other good matches then better have text file or even XML than no matches at all?
If a site has a link to its robots.txt, then it is directing non-bot visitors to it. Makes it fair game for search engines to index.
Not everyone understands what this implies. A lot of things get indexed that surprise people.
We're much more careful now than we used to be, having got a few nasty surprises ourselves.
use a program like teleport and it may be surprising to a lot of webmasters what it finds that they didnt think was accesible. I assume gbot is far more effecient at finding things.