Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- shelob v1.0 - No robots.txt
fiestagirl - 7:00 pm on Jun 17, 2007 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/3366525.htm
According to my records these guys lost their privileges in 4/2006, after scraping with a Python UA.
Brought to you by WebmasterWorld: http://www.webmasterworld.com