Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- Googlebot getting caught in robots.txt spider trap


Staffa - 3:26 pm on Aug 3, 2011 (gmt 0)


Ditto from 66.249.66.79
stepped in my honeypot

Folders have been listed for several years under both:
User-agent: googlebot
and
User-agent: *


Thread source:: http://www.webmasterworld.com/google/4346138.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com