Msg#: 4522318 posted 9:14 am on Nov 25, 2012 (gmt 0)
It just asks for my home page at last 100+ times a day and won't take 403 forbidden for an answer.
This has been going on for many many months now, usually the same IP, day after day, sometimes multiple times in a minute, then every minute, every few minutes, and then it slows down for a few minutes before it starts up all over again.
At this point I'm beyond mystified, I'm just down right annoyed. I'd save a lot of wasted disk space logging this stupid thing if I just dropped them in the firewall but it's kind of amusing at some level.
Maybe I should try randomly 301 and 302 redirecting just their crawler to all sorts of crazy places and see what happens.
I used to redirect some spammers to 127.0.0.1 and that was fun :)
Msg#: 4522318 posted 10:16 am on Nov 25, 2012 (gmt 0)
So did I until I realized that robots don't have to follow redirects. That kinda took the fun out of it :( Except for one batch of Ukrainians who got so wigged out, it really did cut back on their visits.
Someone hereabouts suggested the alternative of redirecting back to the originating IP. Like Return To Sender. If it's a very stupid robot, it might even work.
Msg#: 4522318 posted 10:41 am on Nov 25, 2012 (gmt 0)
I also though about giving them an infinite honeypot and see how many pages they'd grab that contained nothing but a page full of the repeating word "MORON".
Using gzip, assuming they'll accept it, I could compress a gigabyte(s) of data into a very tiny file that when unzipped could cause issues, especially if they unzip a few of them. I did that to someone before but I only did a 1 megabyte page and it compressed down to a couple of hundred bytes but I'd go much bigger for these guys just because they're really asking for it.