Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Google's Robots.txt file

wow, some really weird stuff!

         

internetheaven

9:49 am on May 22, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I was checking out Google's robots.txt file and saw some really weird things:

Disallow: /mac?
Disallow: /microsoft?
Disallow: /unclesam?
Disallow: /katrina?

and so on. Any one got any speculations on why Google have a specialised government search, microsoft search and a search on hurricanes? There were loads more, I just wondered what they could be up to?

g1smd

12:50 pm on May 23, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



www.google.com/unclesam returns ONLY results from .gov and .mil and .us sites - nothing else.

Receptional Andy

12:53 pm on May 23, 2006 (gmt 0)



Google have run these for a long time. Some of them are useful, other ones (elmer fudd [google.com] for example) are just for fun!

For the ones you mention try [google.com...]