Selective hotlinking

using .htaccess

[RammsteinNicCage]

I've tried doing a search for this, but I'm probably not using the right words.... What I want to do is disallow all hotlinking to images, except for three of them (one of the images is in the same directory as many of the others that I do want to disallow).

What I'm thinking about doing first is to make it so that a different image is loaded that asks for the image to be removed from the site or forum. Then, so I don't waste much bandwidth, have no image loaded after a week or two.

So, my questions: how is it possible to allow hotlinking on a couple of images? What happens when I don't have any image loaded - broken images show up? Bandwidth isn't consumed anymore? Error codes?

Jennifer

PS, any good, easy to understand, tutorials for htaccess that anyone would recommend? I've been looking around and mostly see sites that give examples, but don't explain why there are symbols like $, %, ^ all over.

[jdMorgan]

Hi Jennifer,

Chack out our charter [webmasterworld.com] for forum info and for the link to a regular-expressions tutorial that will answer your questions about "hats and dollars", i.e. "^" and "$", etc.

The only tough thing about your proposed plan is the "show no image after a week", as this "week timer" would have to be associated with the image request referrer (the domain name of site w/hotlinked image). You would have to implement a database to track how long each referring site has been responsible for image requests in order to make the decision about whether to serve an alternate image or no image. Unless you really want to do it by hand...

The key to the problem is to make a rewrite rule set to block hot-linking, and then bypass that rule set (using RewriteCond directives) to allow image display to some sites, based on your criteria.

Some bandwidth will always be consumed as long as your server is receiving image requests, whether by serving the alternate image or by serving a 403-Forbidden error response. Just keep your custom 403-Forbidden error page short to minimize this banwidth loss.

Jim

[jdMorgan]

Jennifer,

I got my neurons crossed in the third paragraph:

The key to the problem is to make a rewrite rule set to block hot-linking, and then bypass that rule set (using RewriteCond directives) to allow image display to some sites, based on your criteria.

That should read:
The key to the problem is to make a rewrite rule set to block hot-linking, and then bypass that rule set (using RewriteCond directives) to allow certain images to display for all referrers, based on your criteria. That is, use RewriteCond to bypass the following access restrictions if the %{REQUEST_URI} variable indicates a request for one of your hotlink-replacement images.

Jim

[RammsteinNicCage]

D'oh, I only checked the library, not the charter. :p Thanks for that!

As for the time thing, I was planning on doing that by hand, but maybe I don't have a good enough understanding of how this would work yet.... Let me give you a little more detail:

I have a lot of images in images/ and subfolders of images/. There is one image that is in images/ that I do want to be displayed. So, I'm guessing what I would do is first make all images display as a "do not display" type of image (this would only be one "rule" thingy, right?) and then go back into my htaccess whenever I want to and change it to completely disallow hotlinking. After these rules in the htaccess, I would put in something that would allow this one image to always display - is that right?

Jennifer

[jdMorgan]

Let's handle the simplest variant first.

First write a rewrite ruleset that disallows hotlinking of all images (there are many examples posted here).

In outline form:

If HTTP_REFERER NOT blank AND
If HTTP_REFERER NOT <your domain>
Then
If image directory request, serve 403-Forbidden response or alternate image (as desired), exit mod_rewrite

Next, add an exclusion to bypass that ruleset for one or more specific images. As implemented in mod_rewrite, this "bypass" code would actually precede the anti-hotlinking ruleset:

If REQUEST_URI NOT <always-allowed image URL-path(s)> AND
If HTTP_REFERER NOT blank AND
If HTTP_REFERER NOT <your domain>
Then
If image directory request, serve 403-Forbidden response or alternate image (as desired), exit mod_rewrite

Finally, building on the above to temporarily serve a "please remove" image for some referrering sites, but not others:

If REQUEST_URI NOT <always-allowed image URL-path(s)> AND
If HTTP_REFERER NOT blank AND
If HTTP_REFERER <particular hotlinking site URL> OR
If HTTP_REFERER <another hotlinking site URL>
Then
If image directory request, serve "please remove" image, exit mod_rewrite
- Else (if above ruleset does not apply) -
If REQUEST_URI NOT <always-allowed image URL(s)> AND
If HTTP_REFERER not blank AND
If HTTP_REFERER not <your domain>
Then
If image directory request, serve 403 forbidden response, exit mod_rewrite

The above pseudo-code examples correspond closely with mod_rewrite's RewriteCond and RewriteRule directives, and the first example can be directly-correlated with code in previously-posted threads. This is a pretty good way to work out what you want to do, before getting to the detailed implementation level. The overall If-If-Then-If-Then structure of the code is fixed by the capabilites of RewriteCond and RewriteRule.

Hopefully, these examples will help you get a handle on mod_rewrite. While it may seem quite daunting at first, it is really a very small 'language' but with a lot of precision and power.

Jim

[RammsteinNicCage]

Ok, that makes a lot of sense to me and I should hopefully be able to implement it. :)

I am having one other problem and have one more question. First, the problem:

I have some pages like /news.html, /news1st03.html, /news2nd04.html, etc. I'm moving them into /news/ and renaming /news.html to /index.html. So, the first part is fine with

RewriteRule ^news\.html$ /news/index.html [R=301]

However, the other news pages aren't cooperating. I tried using

RewriteRule ^news(.+)\.html$ /news/news$1\.html [R=301]

(those $1 thingies, whatever they're called - extremely helpful! :) ) and the pages won't load. It seems like it keeps sending the request and occassionally I'll get a 403 with something like /news/news/news/news/news/news/news/news/news/news/news/news/news/news/news4th02.html (but /news/ is repeated a lot more). I've used the same type of line for other pages and they work fine, the only difference between them and the news pages is that the wildcard starts with a number, would that cause this?

Now, the question - when do I want to use an L flag or turn off the rewrite engine? When I'm completely done rewriting?

Jennifer

[jdMorgan]

You are getting recursion...

File news1st.html is requested by the browser.
Your code redirects that request to /news/news1st.html, sending the new URL back to the browser.
The browser resubmits the request, using the new URL as specified by the 301 response.
Your code redirects that request to /news/news/news1st.html.
The browser resubmits the request, using the new URL as specified by the 301 response.
Your code redirects that request to /news/news/news/news1st.html.

This pas-de-deux continues, until either the browser or the server gets tired of it (usually 10 redirects).

The problem is that the pattern "^news(.+)" matches a URL-path starting with "news" followed by "one or more of any characters. So, that pattern will match "news/news1st" just as well as it will match "news1st" (what you expected it to match).

Change your rule to

RewriteRule ^news([^/]+)\.html$ /news/news$1\.html [R=301,L]

and your problem will go away.

This is because you are now asking mod_rewrite to match any URL-path starting with "news", followed by one or more characters which are not a slash "[^/]+", followed by a literal period "\.", followed by and ending with "html". This demonstrates the power and precision of regular expressions, and demonstrates why we go to the trouble of using them -- because they can identify unique patterns with a minimum of coding. The downside is that regex can be a bit obscure until you're used to it.

The $1 thingies are termed "back-references," because they refer back to part of the regex pattern that must be matched to invoke the rewriterule. You may create up to nine backreferences ($1 - $9) to subpatterns in the RewriteRule, and nine more backreferences (%1 - %9) to subpatterns in the last-matched RewriteCond immediately preceding the RewriteRule. The order and precedence of left parentheses determines the numbering, and the parentheses can be nested. a pattern of ^(A(B)C)$ results in $1=ABC and $2=B, for example.

When to use the [L] flag? Always use it unless you have a good reason not to. The main reason you would not use [L] is if you want to rewrite the URL of a single http request in multiple steps. This is almost always unnecessary, and wasteful of server resources. You're not likely to get in trouble soon if you just always use [L] on redirects and internal rewrites; Certain other responses that mod_rewrite can generate don't need it though, because the [L] is built-in. Examples are [G], [F], and [P].

Jim

[viggen]

sorry for hijacking this thread, however it seems to fit

To prevent hotlinking i found a sricpt generator that gave me the following code for my image folder .htaccess, and all seem to work, however i just want to make sure i havent overlooked anything and all is fine with this code and i am not blocking something/someone i shouldnt...

Options +FollowSymlinks
RewriteEngine on
RewriteCond %{HTTP_REFERER}!^$
RewriteCond %{HTTP_REFERER}!^http://(www\.)?you-are-allowed-to-link.com(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER}!^http://(www\.)?you-are-allowed-to-link.com(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER}!^http://(www\.)?you-too.com(/)?.*$ [NC]
RewriteCond %{HTTP_REFERER}!^http://(www\.)?you-too.com(/)?.*$ [NC]
RewriteRule .*\.(gif¦jpg¦jpeg)$ [will-show.com...] [R,NC]

cheers and thanks
viggen

[jdMorgan]

viggen,

I can't tell if this code will work for you or not, since you didn't describe what you want to do with hotlinkers. For example, do you know you are redirecting hotlinkers to an alternative image that is located on another site? If so, then *you* are hotlinking too, and just as guilty as the sites that link to your images. If not, then the RewriteRule at the end of your code should do a server-internal rewrite, rather than an external redirect as it does now.

There is a lot of "fat" in that code, and it could be cleaned up quite a bit. If you are going to use an on-line code generator, it's a good idea to take the code it generates and then use the mod_rewrite documentation and a good regular expressions guide to check it and understand how it works. Don't put code on your server that you don't fully understand, no matter how many "experts" say it's OK -- Doing this repeatedly can create a time-bomb, because when something eventually fails on your server, you will be faced with a huge block of code that you don't understand, and debugging all of the potential interactions between the snippets of code gathered here and there over time can be very difficult.

Jim

[viggen]

thanks jdMorgan for your quick answer,

the alternativ image is located on another site of mine, so i hotlink to myself...(not a good idea?)

basically what i want is (got lots of images that end up on forums and they eat up my bandwidht) that whoever hotlinks gets a tiny image that will save me a couple of GB a month...

I haven't installed the .htaccess yet, (just tested) but wanted to hear first from the expert first ...

when you say "fat", what exactly do you mean with that,
... and yes i will go into the documentation to check it out for myself but would appreciate some directions...

cheers and thanks
viggen

[jdMorgan]

By "fat," I mean there are a lot of unnecessary characters in the regular expressions patterns.

If this were my code, I'd use something like this:

Options +FollowSymlinks
RewriteEngine on
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://(www\.)?you-are-allowed-to-link\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?you-are-allowed-to-link\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?you-too\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?you-too\.com [NC]
RewriteCond %{REQUEST_URI} !^/hotlinking\.
RewriteRule \.(gif¦jpe?g)$ /hotlinking.bmp [NC,L]

-or-

Options +FollowSymlinks
RewriteEngine on
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^http://(www\.)?you-are-allowed-to-link\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?you-are-allowed-to-link\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?you-too\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?you-too\.com [NC]
RewriteRule \.(gif¦jpe?g)$ - [NC,F]

The first version above rewrites hotlink image requests to a local replacement image, and so "hides" the fact that you are using anti-hotlink code.

The second version of the rule simply returns a 403-Forbidden response, and will show a broken image on the site that is hotlinking.

A third alternative, which guarantees maximum compatibility across all browsers, is to replace the hotlinked image with a replacement image of the exact same filetype. This requires you to create several replacement images, but prevents confusing the browser by giving it a .bmp when it asked for a .gif:

RewriteRule \.(gif¦jpe?g)$ /hotlinking.$1 [NC,L]

Furthermore, you may find that the code creates an infinite redirection loop when rewriting to a local replacement image. For that reason, I included an additional RewriteCond to prevent this.

By investigating the differences between the machine-generated code and what I show above, you should learn quite a bit about mod_rewrite and regular expressions. Links to resources on these subjects are available in our forum charter.

Jim

[Warboss Alex]

Err.

Why not put the three images you want to preserve in a seperate sub-directory with its own htaccess file, which overrides the hotlink disallowing?