-- Search Engine Spider and User Agent Identification
---- Anyone know the name of Wayback Machine's robot?
not2easy - 4:18 am on Dec 25, 2012 (gmt 0)
I will second (third?) the advice to block both ia bots. If someone wants to scrape, let them come to the source where it can be minimized. I have the originals stored in backups and on the original hard drives back to 2000 anyway.