anti-hotlink rewrite

[sssweb]

The following snippet is supposed to show a pic of my choice whenever someone hot-links to an image on my site. Can anyone see a reason why it doesn't work?

Options +FollowSymlinks
RewriteEngine On
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://www.mysite.com [NC]
RewriteCond %{REQUEST_URI} !hotlink\.(jpg|gif|png) [NC]
RewriteRule .*\.(gif|jpg|png)$ [domain URL]/images/hotlink.gif [NC]

[g1smd]

In what way does it not work?

If you have previously viewed the right image, your browser will serve the cached image.

Clear browser cache before testing.

Also, you are serving requests for jpg and png URLs with content that is really a GIF. That might not be a great idea.

[sssweb]

Currently, my .htaccess is set up to simply deny hotlink image requests (leaving the usual small red 'x' in an empty image frame on IE browsers). I have the URL of a site that does this, and see the red 'x' on their page. When I update my .htacess file using the above code, hoping to see a pic with my domain name & URL on it ('hotlink.gif'), I still get the empty red 'x'. Tried clearing cache; no change.

Not sure what the issue is with serving jpg/png URLs with a .gif -- can you elaborate?

[jdMorgan]

The code can be cleaned up a bit, but it actually should have worked. Be sure you actually deleted your browser cache (forcing a reload often will not suffice).

Options +FollowSymlinks
#
RewriteEngine On
#
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://www\.example\.com [NC]
RewriteCond %{REQUEST_URI} !^/images/hotlink\.gif$
RewriteRule \.(gif|jpg|png)$ http://www.example.com/images/hotlink.gif [R=302,NC]

Jim

[edit] Fixed typo in 2nd RewriteCond as noted below. [/edit]

[edited by: jdMorgan at 5:03 pm (utc) on Jun 27, 2010]

[sssweb]

That's better, but now it serves up hotlink.gif on MY site, but still not on 3rd party sites. Any idea why?

BTW, the bit that makes it work is the ".\." in:

RewriteCond %{HTTP_REFERER} !^http://www.\.example\.com [NC]

I first thought you had a typo there; why the first period? I understand the backslash and 2nd period.

[g1smd]

It is a typo. It should be \. only.

[sssweb]

Then why when I take that first period out, it doesn't work at all?

[jdMorgan]

Most likely because you are not deleting your browser cache between testing referrers from which the normal image *should* load, and those for which it *should be replaced.*

If your browser loads a cacheable image, it will show you that image until either the image expires, it is replaced in the browser cache because the cache is full and it is the oldest/least-frequently-requested object, or you delete your browser cache.

Testing this kind of stuff requires test/delete cache/change-referrer/test/delete cache/change referrer/test/delete cache... Personally, I turn my browser cache off when doing this kind of work with a Firefox add-on. But you can also turn it off by setting "cache for N days" to zero.

If you are strictly deleting your cache between tests, and if that hostname exactly matches the actual hostname you are requesting, and it still does not work, then there is something very wrong with your server.

Jim

[sssweb]

I think I got it; JD, your code was good, just needed to remove the start anchor in:

RewriteCond %{REQUEST_URI} !^/images/hotlink\.gif$

so that it reads:

RewriteCond %{REQUEST_URI} !/images/hotlink\.gif$

Will apply it as needed and re-post if I have problems.

Thanks for everyone's help.

[jdMorgan]

The question is "why did you have to remove that start anchor?" Is the given URL-path pattern incomplete? -- Is /images itself in a subdirectory?

Jim

[sssweb]

I have one more issue.

My original .htaccess was set up to simply deny hotlink requests by rewriting them to my homepage (which, since it's not a pic file', displays the empty image box as explained in an earlier post). This has the added benefit of redirecting anyone who directly types an image URL into their browser's address bar to my homepage.

Now though, since I'm displaying my hotlink.gif in place of their requested image, users can get to pics via the URL in the address bar.

Any way to combine the two effects? My original rewrite line was:

RewriteRule /* http://example.com [R,L]

I tried including both in the same .htacess file in various ways, but they seem to conflict. One or both rules won't work.

[jdMorgan]

This would leave the originally-requested URL in the address bar. But be warned -- it's even harder to test thatn the redirect version... :)

Options +FollowSymlinks
#
RewriteEngine On
#
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://www\.example\.com [NC]
RewriteCond %{REQUEST_URI} !^/images/hotlink\.gif$
RewriteRule \.(gif|jpg|png)$ /images/hotlink.gif [L]

That's about all I can think of to do -- You can redirect or you can rewrite, but you have to choose one location to redirect or rewrite to, given any particular set of request conditions...

Jim

[sssweb]

When I use your code (and only your code), it displays my hotlink.gif in both situations (a site hotlinks to me or a user types an image URL in the address bar). That works for me since it still protects my other images, even w/o redirecting to my homepage.

Just for my info though, can you explain why this works different than the earlier code? Looks like the only difference is the last line, where you redirect to a relative URL rather than an absolute one. Also the end [L] is different, but I don't see how either of those change the rewrite.

[sssweb]

I take it back; your code works the same as the earlier version -- it allows users to type an image URL in the address bar and view the pic.

I thought there might be a way to include a separate rewrite in the same htaccess file, maybe using different conditions, like:

RewriteCond %{HTTP_HOST} !^http://www.mysite.com [NC] 
RewriteRule /* http://www.mysite.com [R,L]

but that doesn't work. Anything similar to that?

[g1smd]

I thought there might be a way to include a separate rewrite...

But, that isn't a rewrite. It's a 302 redirect.

And again, "doesn't work" is completely useless as a fault report.

[jdMorgan]

If the user types in an image address, the browser does not send a referer header, because there is no referer. The code above allows these no-referrer requests because it must do so in order for your site to work at all with ISPs that place a caching proxy or Web filter between their users and the internet. Examples of these would be all AOL and Earthlink users, and many corporate users.

As a result, no good anti-hotlinking code ever blocks blank referrers, because doing so would make your site look broken to many visitors. This is a limitation created by the fact that referer headers are optional, and that referers from direct type-ins or from network-cached-page requests are meaningless/undefined/blank.

If more "security" is needed, then our usual recommendation here is to use a "cookies and script" method: Here, an "authorizing page" on your Web site --say, the page that includes the image or perhaps the home page-- sets a cookie. All image requests are then rewritten using mod_rewrite to an image-serving script.

So when an image request is received from the visitor's browser, this script is invoked. It checks the cookie and if present, it opens the image file, reads in the image data, and outputs that image data to the browser with correct MIME-type headers, etc.

If the cookie is not set, then the script can return a redirect or a 403 forbidden response, or it can open and serve an alternate anti-hotlinking image file (as in the present method).

The modification that I last made to the code above was to change the original code which invoked an eternal 302 URL-to-URL redirect to so that it became an internal URL-to-filepath rewrite; Image requests with non-blank incorrect referer headers are now internally rewritten to access a different *file* than normal, rather than sending a redirect response to the browser to tell it to GET that alternate file using a new HTTP request. Therefore, the new code does not inform the browser that an alternate image is being served, and so the address bar does not change.

Jim

[sssweb]

Jim, thanks for taking the time to explain that. I gathered from your last paragraph that your code should work, so I re-tested, checking to make sure I copied it exactly. Now it works and I think I figured out what the problem was and why I was getting different results with different tests. It's that start anchor in:

RewriteCond %{REQUEST_URI} !^/images/hotlink\.gif$ 

As I indicated earlier, I had taken it out because that was the only way it seemed to work. Now, in reviewing the posts, I see that you asked why I had to do that. Somehow I missed your post (guess I didn't refresh my page in the 8 minutes between your post and my next one -- sorry). To answer your question, I have two image dirs on my site, one is a sub-directory and one is not. ("Aha!" you say.) Hence the conflicting test results and other probs.

I put the start anchor back in and it looks to be working now (though I reserve the right to re-post if it goes haywire again ;) ).

Thanks for your help, I really appreciate it.

[sssweb]

Now I'm really baffled -- it's not working again! The 3rd party hotlink works fine, but you can still access the image by typing the URL in the address bar. It's not working in both my main image folder and my sub-directory (as mentioned in my previous post). I'm sure it's not a cache issue, because I'm calling different images each time. I even tried it in my Opera browser, aside from my usual IE. My code is:

Options +FollowSymlinks
#
RewriteEngine On
#
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://www\.example\.com [NC]
RewriteCond %{REQUEST_URI} !^/images/hotlink\.gif$
RewriteRule \.(gif|jpg|png)$ /images/hotlink.gif [L]

(Yes, I substituted my real domain for example.com)

Jim, you said, "be warned -- it's even harder to test than the redirect version...." Is there something I should know about testing?

Here's another quirk I discovered; I tried replacing:

RewriteCond %{HTTP_REFERER} !^http://www\.example\.com [NC]

with:

RewriteCond %{HTTP_REFERER} !^http://(www\.)?example\.com/.*$ [NC]

to account for "www." being in the URL or not. When I linked to my image from a third-party site, hotlink.gif displayed correctly if the image link was http://example.com/images/etc, but if I linked to http://www.example.com/images/etc, the pic from my site showed. When I refreshed the page, THEN hotlink.gif showed in both cases. This happened in 3 separate tests on 3 different images.

When I changed the htaccess line to:

RewriteCond %{HTTP_REFERER} !^http://(www\.)?example\.com [NC]

it fixed the hotlink quirk (but not direct access via address bar URL).

I thought the whole thing might be related to htaccess code I have in my top level domain directory, designed to rewrite all http://example.com/etc URLs to http://www.example.com/etc (I was told that's good practice):

RewriteEngine on 
RewriteCond %{HTTP_HOST} !^.*example\.com [NC] 
RewriteRule (.*) http://example.com/$1 [R=301,L]

RewriteCond %{HTTP_HOST} ^example\.com$ [NC] 
RewriteRule ^(.*) http://www.example.com/$1 [QSA,L,R=301]

But when I removed that for testing, nothing changed.

Any ideas?

[sssweb]

Jim said:

The modification that I last made to the code above was to change the original code which invoked an e[x?]ternal 302 URL-to-URL redirect to so that it became an internal URL-to-filepath rewrite; Image requests with non-blank incorrect referer headers are now internally rewritten to access a different *file* than normal, rather than sending a redirect response to the browser to tell it to GET that alternate file using a new HTTP request. Therefore, the new code does not inform the browser that an alternate image is being served, and so the address bar does not change.

Re-reading this, maybe I misunderstood. Are you saying your code is NOT designed to block access via address bar URL? If so, that would (obviously) explain why it doesn't work (but then I'm not sure why you suggested it, and don't know why it seemed to work when I first tested).

Can you please confirm whether it should or should not handle address bar URLs. If it can't, and nobody else here has a solution that handles both the address bar and 3rd party hotlinks, then I'll look into other options.

Thanks.

[sssweb]

As an alternative to resolving the problems with the above code, is there maybe a simpler way to distinguish requests coming from the address bar and those coming from an internal file on my site? Apache lists several server strings recognized by .htaccess, including:

HTTP_FORWARDED
REQUEST_METHOD
SCRIPT_FILENAME
AUTH_TYPE
THE_REQUEST
IS_SUBREQ

Would one of those or some other, used as a

RewriteCond %{...}

, distinguish the two requests?

[jdMorgan]

I detailed this above, but here's the simple version:

In .htaccess, there is no way to block "address bar" URL (typed-in-URL) requests without making your site look totally broken to *all* users of AOL, Earthlink, and many other large and legitimate ISPs and corporations with similar caching-proxy networks.

If you need better "image security," then implement the cookies-and-script solution described above.

Sorry, this stuff is just not as simple as it may appear.

Jim

[sssweb]

I'm probably beating a dead horse but just to be sure, you say that you can't use the empty referrer method because it also blocks AOL users, etc. Fine, but doesn't some different variable like one of those I mentioned, i.e. different than

{HTTP_REFERER}

, distinguish between address bar and internal page requests, AND also between AOL-type requests (which, it seems to me, are the same as any other internal page request except for the referrer issue)?

I don't want to take up more of your time with this - I appreciate the help you've already given - so if the answer is no, a simple "No" will suffice, and I'll look into the cookie solution you suggest. But if it's yes, I'd appreciate some advice on how to use a different variable.

[wilderness]

a simple "No" will suffice

NO