Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- ia archiver
dstiles - 10:07 pm on Jul 14, 2012 (gmt 0)
Real ia_archiver DOES respect robots.txt - haven't seen the real one in years.
Thread source:: http://www.webmasterworld.com/search_engine_spiders/4471054.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com