Need to Block traffic from .ru domains

[aristotle]

I was already getting tired of all these .ru spam backlinks, and now suddenly I'm getting a lot of image fetches from a couple of .ru domains. Can someone please give me some code to add to my .htaccess file to stop this. Thanks

[wilderness]

Can someone please give me some code to add to my .htaccess file to stop this.

That's not how this forum works.
Please read the Forum Charter.

Review this thread [webmasterworld.com]

[aristotle]

Sorry for violating the rules. I just thought this type of block might be so common that someone could easily give me a quick answer. It's been a long time since I changed an .htaccess file, and I don't remeber too much about how it works. Anyway I'll try to find the answer elsewhere.

[wilderness]

There is a valid solution at the end of thread link I provided, although it needs some modification.

[aristotle]

wilderness - Thanks
Yes I saw it at the end of that thread. But I didn't know if it's correct, and I don't have enough confidence or knowledge to verify it on my own. I've always been wary of making changes to an .htaccess file unless I'm absolutely sure it's correct. Anyway, I don't have time to pursue it any more today, so I'll try to get back to it tomorrow. Thanks again for your help.

[lucy24]

I started out with a similar line but later had to tweak it:

RewriteCond %{HTTP_REFERER} \.(ru|ua)(/|$) [NC]

A proper referer would always have a / followed by other stuff-- but I think it's safe to say you also want to exclude forged referers that may end in a naked .ru ;) In general, if the Real Thing is bad, then anything pretending to be that same thing will be as bad or worse.

[aristotle]

I've just done some searching on the web and found the following code in two different articles:

RewriteEngine On
RewriteCond %{HTTP_REFERER} \.ru [NC]
RewriteRule .* - [F]

If I substitute the code that lucy provided, which includes the .ua domain and also blocks forged referrers, then it becomes:

RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(ru|ua)(/|$) [NC]
RewriteRule .* - [F]

So since I don't really understand this type of code too well, can someone please tell me if this is correct.
Thank you

[aristotle]

Well I went ahead and added the new code to the .htaccess file a few minutes ago, and just saw an attempted hotlink image fetch from an .ru domain in my logs. The server response was:

Http Code: 500

I looked this up and found that it means "internal server error". So does this mean that something is wrong? As far as I can tell, nothing else has been affected, and other requests are being fulfilled as usual.

[aristotle]

After some more investigation, I believe the server should have returned a 403 (forbidden) response to the hotlink fetch attempt. But I don't think my hosting company configures its servers to do that. So it gave an internal server error response instead. Does this make sense?

[wilderness]

After some more investigation, I believe the server should have returned a 403 (forbidden) response to the hotlink fetch attempt. But I don't think my hosting company configures its servers to do that. So it gave an internal server error response instead. Does this make sense?

Makes perfect sense, and the same 500 "internal server error" and/or "loop" has happened to most us.
Especially when we have not properly configured two things:
1) Custom 403
2) ErrorDocument 403 with path to 403

[aristotle]

@wilderness
Thanks for your reply. So will these internal server errors cause any problems for my website?

As I said, I don't think my hosting company configures its servers to return a 403 for anything, and that's why it's returning 500..

[wilderness]

There have been some recent (past few months) discussions on this topic between myself, grandma genie, lucy and some others.
Whether it was this forum or the SSID forum, I do not recall.

My host (el cheapo) requires custom 403's and ErrorDocuments to be configured in the CP (Control Panel).
Different shared hosts offer different configuration.

Your host may require CP configuration, or, may simply function with the appropriate htaccess rules for same.

[lucy24]

Once again I have overlapped the preceding 3 posts, but I'll leave it on the GMTA principle ;)

:: peering into crystal ball ::

Do you have a custom 403 document defined with something like

ErrorDocument 404 blahblah.html

? If so, you'll need to make a preliminary RewriteRule that exempts requests for the 403 document. If you don't, it goes like this:

bad robot tries to get file, is blocked
server tries to display 403 document
htaccess says you can't show anything to bad robot, so request for 403 document is blocked
server tries to display 403 document for new block
htaccess says you can't show anything to bad robot, so request for 403 document is blocked
server tries to display 403 document for newest block
htaccess says...

et cetera. Result: 500 error due to infinite "internal redirects" (otherwise known as rewrites). So your bad robot does end up getting locked out-- but not in exactly the way you wanted.

This happened to me when I changed from the default "Forbidden.html" in the top level, to collecting all the error documents in a different directory. I think what happens is that the config file has a global exemption for files called www.example.com/forbidden.html, but if you use anything else, you have to code your own exemption.

[aristotle]

Well lucy, I've never created a Custom403 npage at all for this site. Like I said, I don't think my hosting company configures its servers to return a 403 for anything. I don't know enough about this subject to follow all of your explanation, but I don't think a custom page is the issue.

All I want to do is to block hotlink image fetching from .ru domains. I'm not intersted in custom pages right now. I've never had a custom403 page for this site, and everything has always worked just fine.

Again, here is the code I added to my .htaccess file:

RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(ru|ua)(/|$) [NC]
RewriteRule .* - [F]

Does anyone see any problems with this code, or does anyone have any suggestions for improving it?
Thanks again

[SevenCubed]

Aristotle it's been many months since I messed with config files and I haven't actually looked this up or even tried it but it's simple enough to try it out and remove it if it doesn't work. I'm just trying this from memory. If it does work it's a bare bones solution to simply prevent the 500 error without the need to create custom pages.

Add this above the existing code you have...

ErrorDocument 403 "Niet Commarade, Go Away You Bad Russian Bear"
ErrorDocument 404 "The Page You Were Expecting To Find Has Gone AWOL, Please Try Again Later"

RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(ru|ua)(/|$) [NC]
RewriteRule .* - [F]

[aristotle]

After some further study, it appears that I misunderstood some of the earlier posts in this thread. The word "custom" threw me off, because I thought it referred to a customized replacement for a generic error page. I didn't see how replacing a generic page with a custom page could solve anything. But now I realize that the server apparently doesn't have a generic 403 page, so that something has to be created from scratch.

Another misunderstanding on my part was that I thought my hosting company configured its servers to always return a 404 instead of a 403. That's what I read on its hosting forum about two years ago. But apparently that wasn't correct, or perhaps it was changed when they migrated my site to a new server a few months ago.

So because of these misunderstandings, some of what I said previously probably didn't make much sense, and I apologize for any resulting confusion. And thanks again for your efforts to help me.

[SevenCubed]

Regardless of all that, those 2 lines I suggested here do work. I tried it last night when I got back -- wasn't able to when I posted it. There are 3 ways of handling errors -- 1) With a plain text message, 2) With custom html, 3) I don't remember but in your case #1 will work. And also after rethinking it I think it's probably better to place the 2 new directives after the Rewrite stuff. It doesn't require anything else in settings and you don't have to go looking for any existing pages to modify or create any new ones. The browser will simply display the proper plain text message.

This is all you need...

RewriteEngine On
RewriteCond %{HTTP_REFERER} \.(ru|ua)(/|$) [NC]
RewriteRule .* - [F]

ErrorDocument 403 "Access Denied"
ErrorDocument 404 "Page Not Found"

[aristotle]

@SevenCubed
Yes, i tried your code and it works perfectly! And it apparently only uses 13 bytes of bandwidth. I'm going to use it on a couple of other sites too.

[I didn't use the 404 part of your code because I already have a custom 404 page that contains a link to the home page]

Thanks very much for your help.

[SevenCubed]

Glad it worked and suits your needs. Lucy24 did the real heavy lifting though. Maybe this buys me some "funnies credits" with the mods so...here's one for Lucy...

In keeping with the bare bones and Russian bear theme I should have recommended...

ErrorDocument 404 "Welcome to the Rankin Inlet Outlet Inn, sorry no vacancy, hope you have a safe trip home, watch out for the polar bears"

[aristotle]

Your right. I should have thanked Lucy too, and also wilderness.

[wilderness]

In keeping with the bare bones and Russian bear theme I should have recommended...

You could have added some kind words for the Bolshevik's ;)

[SevenCubed]

The Russians are ok, they are moving in the right direction, it takes time. Some of the smartest technologists hail from there. There are far worst things to be concerned about these days.

[aristotle]

I was just looking at the stats for one of my sites, and just during the week or so since I added this blocking code to the .htaccess file, there have been 873 requests that were forbidden by the server. Nearly all of them are from hotlinks on a couple of .ru domains. To me, this just shows the potential seriousness of the problem, and I feel that this site is a lot less vulnerable now.

I wonder if there are any more domains besides .ru and .ua that should be added to the code for even more protection against future developments. Does anyone have any suggestions? What about .cn ?

[wilderness]

Only you may determine what countries traffic is beneficial or detrimental to your site (s).

My sites deny all countries except US & Canada, however everybody doesn't have the same options and/or preferences.

To add more than a few countries, you reverse the syntax and deny all the 2-letter-domains, and then, make exceptions to allow the countries you desire.

FWIW, these types of questions have nothing to do with Apache coding and/or syntax an belong in the SSID forum, where those forum participants are more aware of the pests

[aristotle]

@wilderness
You're right. It just flowed into my mind as a natural continuation of this discussion. But I see now that you're correct to say it doesn't belong in ths forum.