Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Anyone know the name of Wayback Machine's robot?

incrediBILL - 3:22 am on Dec 24, 2012 (gmt 0)


According to their site [archive.org...] it's still ia_archiver which is also used by a ton of scrapers, many from China.

I have no clue why anyone would want to allow the wayback machine to archive a site as it's a really bad idea IMO, see NOARCHIVE.NET [noarchive.net...] for a few good reasons.

Thread source:: http://www.webmasterworld.com/search_engine_spiders/4530575.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com