Forum Moderators: phranque

Message Too Old, No Replies

Code for various domain suffixes?

Blocking referrer spam from single domain, but several suffixes?

         

FlashDriveDT

3:21 pm on Nov 19, 2010 (gmt 0)

10+ Year Member



I'm pretty new to htaccess writing and recently had to add commands for blocking referrer spam. Unfortunately the domain I need to block operates with several country suffixes, like .ca, .it. So I had written:

RewriteEngine On
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} kharkovforum\.com [NC, OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc\.com [NC, OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc\.es [NC, OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc\.co.uk [NC, OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc\.it [NC, OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc\.ca
RewriteRule .* - [F]

Unfortunately this gives me internal server errors so my page doesnt load (with or without # Options +FollowSymlinks). Im suspecting that this is either because
a) its the same domain over and over, just with different suffixes. Is there any way to shorten this to one line, with a variant for all different county suffixes?
b) there is no www from this domain, only "vi". The full domain is "vi.ebaydesc.co.uk", not "www.vi.ebaydesc.co.uk". But maybe htaccess mistakens that?

Thanks for all your help!

FlashDriveDT

3:25 pm on Nov 19, 2010 (gmt 0)

10+ Year Member



Also, BEFORE these entries I have hotlinking prevention. Would it be better to add this AFTER the block referrer spam entries? Or no difference? E.g.:

RewriteEngine On
RewriteCond %{HTTP_REFERER} !^http://(.+\.)?usbmemorysticks\.net/ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteRule .*\.(jpg|jpeg|gif|bmp|png)$ [img215.imageshack.us...] [L]

RewriteEngine On
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} kharkovforum\.com [NC, OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc\.com [NC, OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc\.es [NC, OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc\.co.uk [NC, OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc\.it [NC, OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc\.ca
RewriteRule .* - [F]

or the other way around?

wilderness

4:36 pm on Nov 19, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



"RewriteEngine On" is only used once per file, unless and only, if you may turned it off for some reason (unknown) within same file.

RewriteCond %{HTTP_REFERER} vi\.ebaydesc [NC]

should function and replace your five lines.

As to the order of sections?
It depends whether your Rewrite is external or internal.

Serving up an alternative image and possibly another website for inline-linking is a bad practice. It's more efficient to simply deny access.

FlashDriveDT

8:47 pm on Nov 19, 2010 (gmt 0)

10+ Year Member



Thanks for the reply, wilderness. So would this be better?

RewriteEngine On
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} kharkovforum\.com [NC,OR]
RewriteCond %{HTTP_REFERER} vi.ebaydesc
RewriteRule .* - [F]
RewriteCond %{HTTP_REFERER} !^http://(.+\.)?usbmemorysticks\.net/ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteRule .*\.(jpg|jpeg|gif|bmp|png)$ - [F]

Note that I put it all together. Would it work this way?

Also, you said "As to the order of sections?
It depends whether your Rewrite is external or internal. "
Can you elaborate on that please? I do have 301 redirects in my htaccess as well, but they are in an earlier part of the file. In between are a few W3 Total Cache plugin entries (the plugin does those automatically this way).

Last but not least, can refer blocks like above slow down my server in any way, or cause an unnecessary load on my server memory? I do get a lot of hits from the ebaydesc sites as people there have posted images from my site.

wilderness

10:15 pm on Nov 19, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



you need to escape the period following vi

vi\.ebaydesc

Jim provides this explanation on externals [webmasterworld.com]

FlashDriveDT

4:31 pm on Nov 20, 2010 (gmt 0)

10+ Year Member



Ok, but the rest of my commands is good, right? I've tested it out and it does seem to work but one thing bothers me. On the ebaydesc sites (for instance) the images do not show like theyre supposed to - BUT when I right click on the broken images in firefox and choose "view image" the image loads fine in the browser. I though that it should serve me a 403 forbidden? Or is that only when the image is also an actual link to the image location that you wont be able to follow that link then? Is this a firefox thing only that right click, "view image" still works?

wilderness

4:57 pm on Nov 20, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



It's likely that using FF in that manner is making a direct request for the image, rather than a referral from the other site.

FlashDriveDT

2:02 am on Nov 21, 2010 (gmt 0)

10+ Year Member



So what do you propose what I should do better?

wilderness

2:18 am on Nov 21, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



"better" than what?

You cannot prevent direct requests for images, unless you serve them all up through a server-side script.

Thus far, you've made a mere two changes as compared to your initial inquiry:
1) Changed the five lines to one line.
2) rather than directing the requests to another image, your simply denying the requests.

Are these adjustments not working in some manner that is unacceptable?

FlashDriveDT

2:32 am on Nov 21, 2010 (gmt 0)

10+ Year Member



Well, I dont understand why firefox allows the right click viewing of the image on those blocked websites - served from my image url. I though that would be blocked too.

Btw, I have just noticed a couple of things that may be wrong in my htaccess. I would appreciate some feedback on that.
Does each "rewrite engine on" also require <IfModule mod_rewrite.c> before and a </IfModule>? after.? And does it make any difference for 301 redirects whether the L is before or after the R=301? I just saw that in some tutorial with the L before the R.

Example:

RewriteEngine on
RewriteRule ^bla-blak$ / bla-blub [R=301,L]
RewriteRule ^bli-bla$ / bli-blub [R=301,L]

vs

<IfModule mod_rewrite.c>
RewriteEngine on
RewriteRule ^bla-blak$ / bla-blub [L,R=301]
RewriteRule ^bli-bla$ / bli-blub [L,R=301]
</IfModule>

I have a few rewrite engines without the ifmodule part and Im wondering if that causes any trouble. The redirects and all work fine but if my htaccess can be optimized I would like to go for it.

wilderness

3:30 am on Nov 21, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Somebody else will need to assist you, I've never used the "IfModule".

as for the "301" and flag issue, there are hundreds of examples of "L,R=301" in the archives.
[google.com...]

Jim has explained it many times and offered the following in June 2010 [webmasterworld.com]"
A redirect is a URL-to-URL translation. It says to the client (browser or search engine robot) "The resource you have requested has moved. Please ask for it again at this new URL." This server response terminates the current HTTP request, and the client must begin a new one, using the URL provided by the server in its redirect response.

Further, search engines will take a 301 redirect as a request to discard the original URL and use only the new one for search listings and ranking.

A rewrite is a URL-to-server-filepath translation, and occurs completely 'inside' the server, and in the context of the current (original) HTTP request. All it does is to change the server filepath associated with the requested URL, and its operation is not visible to the client.

jdMorgan

3:57 am on Nov 30, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Access control by referrer is never 100% effective, because referrers can be spoofed on the one hand, and are often omitted or blocked on the other. So you don't always get a referrer, and if you do, it could be faked.

Your "view image source" requests in Firefox are sent with no referrer, since the images are indeed not being loaded by the original HTML page. Therefore, your rule allows the image to be served (as it should).

Also be aware that while testing rules like this that can operate in multiple "modes" (in this case block or do not block image requests), you must delete your browser cache between tests. Otherwise, your browser will show you stale previously-cached server responses and content, making your test results invalid and confusing.

The only tweak I see to improve your code at this point is to substitute "jpe?g" for "jpg|jpeg" -- It's the same thing, just shorter.

jim