Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Psycheclone


Pfui - 4:30 am on Jun 25, 2006 (gmt 0)


McCOLO again, but hitting a different site where I'd not blocked the heck out of its IPs and UAs --

208.66.195.2
psycheclone
06/24 13:06:47 /robots.txt
06/24 13:07:17 /

So much for the 'let them all see robots.txt even if they're Disallowed' theory. Oh, well!


Thread source:: http://www.webmasterworld.com/search_engine_spiders/3269.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com