Page is a not externally linkable
- Google
-- Google News Archive
---- Googlebot: Deepbot and Freshbot FAQ and Information


lorax - 8:03 pm on Feb 10, 2003 (gmt 0)


Lovely post!

It should be noted that the spider trap mentioned above is an unintentional one. There are intentional spider traps which are used to trap bad bots and other agents a webmaster does not want crawling thier website. You'll find both are talked about here on WebmasterWorld.


Thread source:: http://www.webmasterworld.com/google_archive/9213.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com