Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
malachite - 7:54 pm on Jun 2, 2006 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/3268.htm
Turned up on one of my sites yesterday, and on two more today. Can't find any real info on it, so denied it. It doesn't seem to visit robots.txt, just dives straight in at random, one page per IP.
Brought to you by WebmasterWorld: http://www.webmasterworld.com