Is there a benefit to blocking hack bots on a static website?

Ban them in .htaccess, or just let them get the 404?

[MichaelBluejay]

Every day bots try to look for ways into my website by requesting files with known vulnerabilities (e.g., "/FCKeditor/_whatsnew"). I don't have any such files because my site is static and I don't use any plugins or off-the-shelf software. I'm irked by others trying to exploit my site so I thought I could write a script to scan the log files and then ban any IP in .htaccess which requests such phantom files. (I can't add to the apache2.conf / http.conf file because my webhost overwrites any changes I make to that file.)

However, I don't know if banning the offenders is really any better than just letting them get the 404. Adding a gazillion banned IPs means the server has to parse that list for each and every request by legitimate users, while not banning means that the bot just gets a 404 for their request and no payoff, and it doesn't take the server any more time to return a 404 than to return a 403-Forbidden. But maybe I'm missing something and there's some benefit to banning the offending bots instead of just letting them get the 404. Am I missing something?

[tangor]

I don't ban the ips, I deny the requests. Cleaner, faster, and my 403 response is far smaller than my 404.

YMMV

Note: if static, ie, not PHP, denying .php goes a long way for silence!

That's my benefit.

[dstiles]

In my experience almost all IPs used are one-offs, generally hacked. I have a list of bad words which I reference in PHP to block such calls. But if it's a static site then, as tangor says, block all php - also asp and everything except your own file extension. If the site is not too big then allow only your known file, rejecting all else. If using PHP then block everything else.

[robzilla]

I've never bothered with IP blocking, particularly not after the fact. You could block certain IP ranges but as you say there's always going to be a bit of processing involved; hard to say which option would be more resource intensive, probably won't matter all that much. Serving 404s adds a bit of noise to your log file (neglible after they're gzipped), but is low maintenance and effective enough.

[MichaelBluejay]

Okay, I'm sold on banning requests by file type and/or filename instead of IPs, thanks!

About this:

Serving 404s adds a bit of noise to your log file (neglible after they're gzipped)

Well, a 403 gets logged too, right?

[lucy24]

Well, a 403 gets logged too, right?

It should. Some very large sites set up a firewall so that unequivocally unwanted requests are intercepted before they even reach the server, but that wouldn't apply here.

There is an Option C, though: for certain types of requests, like the ones described in OP, you could return a manual 404. (Flag [R=404] if it's done with a RewriteRule.) This saves the server the trouble of looking for the file, but gives the visitor no information.

[robzilla]

Well, a 403 gets logged too, right?

True, I forgot we're in the Apache forums. I would have the firewall take care of IP-based blocking, but I see you don't have a lot of options besides .htaccess. It could potentially make your .htaccess very large, I think that would be a major downside to that route.

[lucy24]

It could potentially make your .htaccess very large

If you're careful to get the full CIDR ranges, blocking nothing less than a /20 (IPv4) except in extraordinary circumstances, it shouldn't end up that enormous.

But if you feel up to it, header-based blocking is also very useful, because it zaps almost all unfamiliar robots. In time, of course, this too will become less useful, just like checking for “Mozilla” at the beginning of the UA has become well-nigh useless over the last decade or two :(

[tangor]

If you pick and choose what agent, referer, request you block you can catch an extremely large amount of garbage. Including ips I do deny, my entire .htaccess is only 389 lines (including blanks).

[dstiles]

> nothing less than a /20 (IPv4)

A large part of Apnic and quite a lot of other ranges are now /22. Nasty servers such as serverion are often only /24. But otherwise I add obnoxious ranges of /16 + to iptables.

[tangor]

The OPs question was:

Is there a benefit to blocking hack bots on a static website?

It seems to me that denying the requests of the HACK bots is a bit cleaner and faster than 404s. All the other stuff mentioned is also good commonsense management of website access, just some of it might be overkill.

Hack bots are asking for files prone to hacking, and that number of files (ie FCKeditor, et al) are a lot smaller than ip ranges in general.

Meanwhile, having both considerations will go a long way towards finding the desired UNDESIRED traffic, with only the IP side requiring punching holes to allow "known" humans.

YMMV

It's not a perfect world, so there are no perfect solutions.

[Martin Potter]

I have had the same problem as OP, with many requests for wp-login.php and a lot of other files not found on my site. About a month ago I started to give the requesters a special message by using R=403 but declaring

ErrorDocument 403 /forbidden4.php

in my htaccess file and with the referenced php file giving the requester the message "Forbidden. Your IP address has been recorded. " The script also sends the requester's IP address to a special log file, along with some housekeeping info (date/time, the requested filename, UA).

Well, the requests continued and then seemed to have dropped off. In fact, for the last two days there have been so such requests at all! Did the message finally get spread around (unlikely) or am I just seeing statistical noise (probable)?

[dstiles]

Coincidence, I suggest. I've had only a few wp's in the past week or so.

Bear in mind that most of this type of hit originate from compromised IPs which may well become uncompromised later that day/week/month. I think the forbidden file is not really useful except, perhaps, for your own logging reference.

Having said that I do send such IPs to be blocked by iptables and then clean them out every few months.

[Martin Potter]

> Coincidence, I suggest.

You are probably right, given your similar experience.

Requests here for wp-login.php have dropped to something like one or two per week, although I still, and increasingly, get requests for things like /wp-includes/wp-system-cloud.php , and many others like it, every day. The number of variations is surprising, at least to me. It is clear that some hackers make a concerted effort to find vulnerabilities in large and complex websites. (I preferred the days of the script kiddies.)

[Sgt_Kickaxe]

Assuming wordpress is behind the static content and, like 99%+ of webmasters, you're not developing an app on your intall you may want to disable wordpress JSON endpoints too, especially the author endpoint because it gives up the main login name. They only need to guess the pass once they know the main unchangeable login name.

As for the pass, you can .htaccess 403 the wp-login.php file with only your(and anyone else you grant access to) IP whitelisted. Blocking access to wp-login.php for everyone but yourself goes a long way to preventing automated brute force login and DDoS attacks.

Don't forget about disabling the XML-RPC remote stuff too unless you need the functionality of posting via email etc. Make the script kiddies work for it.

Of course, if your install login is "admin" nothing can be done to make it harder to guess.

Not using wordpress? Same guidance, make the important pages inaccessible to all but you by whitelisting yourself, blacklisting everyone else without blocking the visitors you want is... tedious.

Lastly, if you're using an apache server, make sure to create a 403 file and place it in your root folder. Then reference it as an error document in your htaccess (ErrorDocument 403 403.php) above your file control entries, preferably at the top of the file depending on what's in there.

Reason: The 403 page isn't automatically created outside wordpress and instead of a 403 access denied the naughties will get 5x 302 redirects in a row and a message the site isn't redirecting properly. You don't want that extra server load.

[Martin Potter]

Thanks for all that good advice! At present I am not using wp but have printed your comments and put it in my wp folder for when I do need it.

I didn't know about declaring ErrorDocuments near the top of .htaccess. Thanks for that too. In fact, that is where it is, but it might have happened differently.

My wife tells me stories about the troubles that her friends and colleagues have with passwords. It is amazing, and a little frightening. No wonder there are so many successful hackings. Of course it will never happen here. (He said, confidently.)

[lucy24]

declaring ErrorDocuments near the top of .htaccess

Just to clarify: The server doesn't care where the ErrorDocument declaration is. Heck, you could shove it into the middle of a bunch of RewriteRules and nothing would change. The recommendation to put it at the top is purely for your own sanity: any simple, single-line rules go at the beginning where it is easiest to find them. (Another example is the Options directive.)

Edit: In particular, it makes no difference whether the ErrorDocument directive comes before or after access-control rules, since each module is an island. You do need to poke a hole for your 403 document in each module that can issue a 403 (which ones will depend on your site).

[Martin Potter]

Good point, Lucy, thanks. I endorse and adopt anything that preserves sanity!

[Sgt_Kickaxe]

Thanks Lucy, I'm aware it doesn't matter where an ErrorDocument goes in the htaccess file but keeping it tidy, to me, matters a great deal. My error documents are all declared at the top of the file, obviously. My htaccess file is broken down into sections with ###### This Section Is For.... ####### around them which you also probably don't need. If I could color code sections I would.

The page gives me OCD.

Regardless, make sure you have a 403 permission denied file in the root folder. Without it some servers will not redirect properly(302 loop) in some situations, like when you try to block access to your worpress admin folder or login page. You'd be surprised at how many sites block access, whitelist their IP and then never see the redirect loop.

Trying to find out how to fix the redirection error by looking it up on Google is relatively futile, you'll see too many "fix the wordpress 403 permission denied error" refering to an entirely different problem.

Create the file, declare the file, stay sane friend.

[Martin Potter]

"Thanks, Sarge, good advice!" And sanity forever! (When did sergeants ever NOT give good advice!?)

[lucy24]

When did sergeants ever NOT give good advice!?

Just watch out for the “You kiss your mother with that mouth?!” variety of sergeant.

[tangor]

Heh... when I went through Basic and some fool shouted that out from the ranks (thus "invisible") the Sergeant barked back: "My mother is NOT on this parade ground! YOU ARE! Drop and give me twenty!"

Not pertinent to the OP...

The overall answer to the OP regarding "benefit" is what is beneficial to the OP? I find it easier to filter out 403 reading logs than dealing with a bunch of requests for files that do not exist, and will never exist, on my system. Taking the request out is far easier and less intrusive to IP ranges and avoids having to punch holes in IP ranges for valid traffic.