Forum Moderators: phranque

Message Too Old, No Replies

hotlink protection doesn't work

         

Amanda

6:56 am on Aug 3, 2006 (gmt 0)

10+ Year Member



This is so frustrating because I have a lot of images on my site and there are a lot of people direct linking to my images and it's eating up my bandwidth. I tried to use an .htaccess file to stop direct linking, but it won't work! I made sure to clear my temp files before I checked and the images still showed up! What can I do besides contacting each and every one of these hotlinkers? I noticed a lot the hotlinking is coming from xanga sites, is there any way to block xanga sites from direct linking?

This is the code I tried in my .htaccess file and I know mod_rewrite is enabled on my server because I checked with phpinfo.

RewriteEngine on
RewriteCond %{HTTP_REFERER}!^$
RewriteCond %{HTTP_REFERER}!^http://(www\.)?websiteurl.net(/)?.*$ [NC]
RewriteRule .*\.(gif¦jpg¦jpeg¦bmp)$ - [F,NC]

[edited by: Amanda at 6:58 am (utc) on Aug. 3, 2006]

jdMorgan

2:25 pm on Aug 3, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You can clean/speed up that code up quite a bit, but I suspect that's not the problem:

RewriteEngine on
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://(www\.)?websiteurl\.net [NC]
RewriteRule \.(gif¦jpe?g¦bmp)$ - [NC,F]

Replace the broken pipe "¦" characters above with solid pipes before use; Posting on this forum modifies them.

In both your original code and this cleaned-up version, the first RewriteCond *allows* access using a blank HTTP_REFERER header. This is necessary in order to prevent blocking legitimate visitors who visit your site through a corporate or ISP caching proxy or those who are running "Internet Security" software, which is blocking transmission of the referer header - most often without their knowledge.

As a result, hotlink control using HTTP_REFERER is only effective enough to stop most cases -- and it makes the hotlinking site look broken, so sometimes the hotlinks get removed. However, it cannot and will not stop hotlinking completely, and you;ll need a more-sophisticated solution if you really need to do that.

You may also want to try a simple test to make sure mod_rewrite is actually *working* -- something like this:


RewriteRule ^test_mrw\.html$ /path_to_a_page_that_exists.html [L]

With this rule in place, request "/test_mrw.html" from your server, and you should see the content of the page that exists.

This search [google.com] wlll lead you to much more information.

Jim

Amanda

6:11 pm on Aug 3, 2006 (gmt 0)

10+ Year Member



Yeah, I tested the mod_rewrite like you said and it worked, but the hotlink protection doesn't. I used the code you posted and it still didn't work, I can see the images that are being hotlinked.

jdMorgan

6:20 pm on Aug 3, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Look at your raw server access log file. Do your own requests for images referred by the hotlinking page have an HTTP_REFERER field in the log? If so, then you have a code problem -- see your raw server error log for possible indications.

If you see few or no logged image requests from your testing session, then your browser, network, or ISP is caching those images, and therefore they aren't being requested from your server, so your code can have no effect.

And finally, if you see no referrer in the logged accesses, then your browser, internet security softwre, or a network caching proxy is blocking the referrer, so again your code won't act.

Again, referrer-based anti-hotlinking is only an easy partial solution, so don't expect it to be 100%. However, if your browser sends a referrer from the hotlinking page, and it actually gets to your server, and your code is working, then the access will be blocked. You will have to repeatedly flush your browser cache (or disable it) during testing; Once that image is loaded and cached, it will appear for all subsequent requests until you flush it out again.

Jim

Amanda

4:03 am on Aug 4, 2006 (gmt 0)

10+ Year Member



I downloaded my raw access logs but I can't open the file because it's too big, so I don't know how else to check the logs.

Is there any way to just block certain sites from hotlinking?

[edited by: Amanda at 4:07 am (utc) on Aug. 4, 2006]

jdMorgan

1:44 pm on Aug 4, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The code above blocks all but your own site from referring image requests, as long as a referer header is sent with the request.

You can easily change the code to block certain sites, again if a referrer header is sent, but if the code above won't work, then any variation of it is also unlikely to work.

So, then you're back to troubleshooting, and you'll need to look at the log files... Back to square one.

How big is this 'unopenable' log file, what program are you trying to open it with, and what format is it in?

Jim

Amanda

5:44 pm on Aug 4, 2006 (gmt 0)

10+ Year Member



The log file is a NET file and I've tried to open it with Notepad and Wordpad.

jdMorgan

6:45 pm on Aug 4, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



A NET file? You're not on a Windows server are you?

Jim

Amanda

3:48 am on Aug 5, 2006 (gmt 0)

10+ Year Member



No - from my cpanel it says Operating system Linux

jdMorgan

2:35 pm on Aug 5, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Apparently, today is my day for mysteries...

I don't know what to tell you, since your efforts to debug this problem are defeated by lack of sufficient tools.

Something's just not right, but I can't figure out what it is without more information, and you can't get that information because the logs are too big (and maybe in a strange format as well).

However, by your request, the following code can be used to block *specific external sites* from hotlinking:


RewriteEngine on
RewriteCond %{HTTP_REFERER} ^http://(www\.)?bad_site\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?another_bad_site\.co\.uk [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?last_bad_site\.net [[b]NC[/b]]
# Critical note: No [OR] allowed on last rewritecond before rewriterule!
RewriteRule \.(gif¦jpe?g¦bmp)$ - [NC,F]

Replace the broken pipe "¦" characters above with solid pipes before use; Posting on this forum modifies them.

This code blocks requests referred by specific sites, whereas the previously-posted code blocks requests referred by any site *except* your own. This code suffers the same shortcoming as the previous -- It can do nothing to help you if the HTTP referer header is blank.

If this code works and the previous code doesn't, that indicates a serious problem with your Apache server installation or with the regular-expressions library installed with your OS.

Jim

slugworth

3:20 pm on Aug 6, 2006 (gmt 0)

10+ Year Member



The ".net" file suffix is coming from the download of the log file from your web hosting control panel. Change the file suffix to ".txt" and open it with any text file editor/viewer capable of working with large files.

Notepad may choke on your log file, try something like Editplus [editplus.com].

jtbell

6:26 pm on Aug 6, 2006 (gmt 0)

10+ Year Member



I tested the mod_rewrite like you said and it worked, but the hotlink protection doesn't. I used the code you posted and it still didn't work, I can see the images that are being hotlinked.

Just a wild guess, but are you testing it by re-visiting the same pages (with hotlinks) that you were looking at before? In that case, the hotlinked images are in your browser's cache so your browser doesn't even need to get them from the server. Try clearing your browser cache.