It just asks for my home page at last 100+ times a day and won't take 403 forbidden for an answer.
117.81.213.16,"ia_archiver","/index.html"
This has been going on for many many months now, usually the same IP, day after day, sometimes multiple times in a minute, then every minute, every few minutes, and then it slows down for a few minutes before it starts up all over again.
At this point I'm beyond mystified, I'm just down right annoyed. I'd save a lot of wasted disk space logging this stupid thing if I just dropped them in the firewall but it's kind of amusing at some level.
Maybe I should try randomly 301 and 302 redirecting just their crawler to all sorts of crazy places and see what happens.
I used to redirect some spammers to 127.0.0.1 and that was fun :)