Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Zeta Interactive?


incrediBILL - 5:53 pm on Sep 18, 2009 (gmt 0)


Robots.txt only supplies directives for polite robots.

.htaccess bans impolite robots that ignore robots.txt


Thread source:: http://www.webmasterworld.com/search_engine_spiders/3991090.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com