Forum Moderators: phranque

Message Too Old, No Replies

Extending the time a detected bad user-agent is rejected

Or a detected bad activity/URI/referer/etc

         

dstiles

5:26 pm on Jul 16, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



On detecting a bad "bot" I would like to prohibit further access from the IP for a period of time. On my windows server I can set a timeout of, for example, 24 hours during which an IP is blocked due to a bad UA, bad behaviour etc.

Ideally, detecting such a thing on one site should block the IP for all sites on the server.

Reasoning: on having its UA rejected, the perpetrator can change the UA - this is not unknown. If a client is detected falling into a bot-trap it should remain in the trap even when fetching a good page afterwards.

I would like to do this on apache (linux) using setenv, rewrite or something slmilar. Is this possible?

not2easy

6:24 pm on Jul 16, 2019 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



In a very old (2004) thread here there is a script to block bots that ignore the "no-follow" added to a link. I don't know that the entire script is still functional but I (who can't php my way out of a paper bag) have used parts and pieces of it to automatically block requests for wp-login.php on non WP sites and it does work super for that. It sends an email so I know there is something to deal with, and where it is. Just suggesting that you may find some useful ideas in that oulde thread to complement your bot traps: [webmasterworld.com...]

At least until a more competent person shows up.

dstiles

10:09 am on Jul 17, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Thanks, I'll take a look.

dstiles

9:37 am on Jul 18, 2019 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Ok, that's pretty much what I do on my Windows ASP server - on-page traps, links in comments etc. Difference is I use mysql to hold the IPs and add a timout to auto-release the URL after a day, with an incrementing count for each new hit to extend the timeout.

I was rather hoping for an apache solution that removed the reliance on mysql.

I did think it was a long-shot. Thanks, anyway. :)