Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- Googlebot getting caught in robots.txt spider trap

lucy24 - 2:30 am on Sep 5, 2011 (gmt 0)


It's a common misconception that robots.txt completely blocks out Googlebot.

Not in this forum it isn't ;) The question is really: Are we stupid or is google crooked?

Thread source:: http://www.webmasterworld.com/google/4346138.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com