homepage Welcome to WebmasterWorld Guest from 54.161.190.9
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
attempt to block hot-linking actually blocks everything
My .htaccess file either blocks everybody or nobody.
stapel




msg:1521607
 3:05 am on Sep 21, 2002 (gmt 0)

I would like to block hot-linking, so I included the following in my .htaccess file:

#Prevent hot-linking of images:
Rewriteengine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://www.domain1.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://domain1.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://www.domain2.com/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://domain2.com/.*$ [NC]
RewriteRule .*[Jj][Pp][Gg]$¦.*[Gg][Ii][Ff]$ [domain1.com...] [R,L]

It worked, in that nobody could access the images. But it didn't work, in that local calls (for instance, to [domain1.com...] which calls domain1 pictures from a domain1 page) returned a 403 error. Not only were the pictures not diplayed, but the page wasn't even displayed.

So clearly I'm doing something massively wrong.

If I can get this figured out, I'd like to try for my real goal, which will be blocking people from downloading my site with FrontPage, et al. I plan to upload this:

#Prevent educators snagging site:
Rewriteengine on
RewriteCond %{HTTP_USER_AGENT} ^.*Copier.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Downloader.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FrontPage.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Microsoft.URL.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Plucker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Reaper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Stripper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*WebVCR.*$ [OR]
RewriteRule /* [domain1.com...] [L,R]

But due to my, erm, less-than-stunning success so far, I'm a bit leery of doing this yet.

Any advice/URLs would be appreciated. Thank you.

Eliz.

P.S. My error redirects (for 404 and 401, though not, for some reason, for the 403) are working properly, so I know the .htaccess file is working to some degree. *sigh*

[edited by: DaveAtIFG at 4:16 pm (utc) on Sep. 23, 2002]
[edit reason] Changed to "generic" URLs [/edit]

 

akogo




msg:1521608
 3:14 am on Sep 21, 2002 (gmt 0)

Excuse me if this is sounds too obvious, but does your server have mod_rewrite? I tried this once only to discover I didn't have mod_rewrite on the Apache server my site was hosted on.

jdMorgan




msg:1521609
 5:35 am on Sep 21, 2002 (gmt 0)

stapel,

Your regular expressions need a few tweaks...

#Prevent hot-linking of images:
RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?domain1\.com/ [NC]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?domain2\.com/ [NC]
RewriteCond %{HTTP_REFERER} !^http://216\.239\.(3[2-9]¦[45][0-9]¦6[0-3])\..*(www)?domain1\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://216\.239\.(3[2-9]¦[45][0-9]¦6[0-3])\..*(www)?domain2\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://babel.altavista.com/.*(www)?domain1\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://babel.altavista.com/.*(www)?domain2\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://216\.243\.113\.1/cgi/
RewriteCond %{HTTP_REFERER} !^http://search.*\.cometsystems\.com/search.*(www)?domain1\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://search.*\.cometsystems\.com/search.*(www)?domain2\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://.*searchhippo\.com.*(www)?domain1\.com [NC]
RewriteCond %{HTTP_REFERER} !^http://.*searchhippo\.com.*(www)?domain2\.com [NC]
RewriteRule \.(jpeg?¦jpg¦gif)$ [domain1.com...] [NC,R,L]

If you have mod_rewrite installed (and I didn't add any typos), this should work.

Note that if you'd rather use a .gif file as your replacement image, you can do so by adding one more RewriteCond:

RewriteCond %{REQUEST_URI} !^/hotlink\.gif$

In other words, once you've redirected to the "buzz off" graphic, you don't want to block it by redirecting again - this would cause an infinite loop, and likely take your server down!

The extra lines I added are to allow Google, GigaBlast, SearchHippo, AltaVista, and CometSystems search engines to properly display their translated and/or cached versions of your site. If you don't want them to be able to display your images with your translated/cached page, just remove those lines. Leaving them in will mean you'll need to keep an eye on these lines in case the search engines change IP addresses or cgi paths in the future. However, you'll be no worse off with incorrect addresses in those lines than if you leave them out entirely now...

You can also save yourself a lot of duplicate RewriteConds (and work) if you are willing to pick a "standard" URL and stick with it. Simply redirect any incoming requests for the "non-standard" domain names to the "standard" domain name. For example:

RewriteCond %{HTTP_HOST} !^www\.domain1\.com$
RewriteRule ^(.*)$ [domain1.com...] [L,R=permanent]

This would "correct" the URL in any browser or search engine that followed a "non-standard" domain link, and eliminate the need to handle all four domain name variants as was necessary in the rewrite rules above. Users would then only be bookmarking your "standard" domain name, and search engines would list only that "standard" domain name, "concentrating" the Google page rank in one listing. I don't know if you've been following the "multiple domain names, one site" threads here recently, but I prefer to just have one domain name "out there" per site and get all incoming links pointing to that one.

Also, this rewrite takes care of all the upper/lowercase variants as well, eliminating the need for all of those [NC] flags in the rewrite rules above.

Just those two lines alone would make a pretty good test to make sure that you host is allowing you to use mod_rewrite; Put them into .htaccess, along with the "RewriteEngine on" directive, and then use a different domain to access your site - e.g. leave off the "www.". You should see the URL in your browser address bar update to show the "standard" URL which includes the "www." in this example.

For more info, see the Apache mod-rewrite documentation [httpd.apache.org] and this handy regular expressions guide [etext.lib.virginia.edu].

Hope this helps,

Jim

[edited by: DaveAtIFG at 4:28 pm (utc) on Sep. 23, 2002]
[edit reason] Changed to "generic" URLs [/edit]

Boule de poils




msg:1521610
 8:59 pm on Sep 21, 2002 (gmt 0)

Hello,

I'm having some similar problems with hot-linking too, except that:
- unauthorized pages do show the default picture for hot-linking
- authorized pages from my own domain name show a forbidden page.

This is rather curious, since I can access my site through another address (http://members.hostname.com/username) and the .htaccess is working fine when I filter access based on this address instead of the domain name (i.e. I can load the actual pictures)

To sum up:

[mydomain.com...] --> Forbidden page
[anotherwebpage.com...] --> hotlink.gif picture

I have been trying for hours now, and nothing seems to work. Has anyone any idea?

Slade




msg:1521611
 9:22 pm on Sep 21, 2002 (gmt 0)

jdMorgan:

Current Message:
Your code works perfect. As it always has according to my searches here... It just does the opposite of what I wanted it to do. I'm playing with domain.tld versus www.domain.tld on my personal site. If anyone's interested in doing that, use this:

RewriteCond %{HTTP_HOST} ^domain\.tld$
RewriteRule ^(.*)$ http://www.domain.tld/$1 [L,R=permanent]

Original Message:

My mod_rewrite knowledge is rather well, non-existant...

Would your two lines for cleaning up domain names mess up references to https? Or, is there an extended solution that will capture for that?

edit:
hmm... doesn't seem to work for me... (i replaced my domain name for his, that was it though)


jdMorgan




msg:1521612
 12:44 am on Sep 22, 2002 (gmt 0)

Slade,

From my first post:

RewriteCond %{HTTP_HOST} !^www\.domain\.tld$
RewriteRule ^(.*)$ [domain.tld...] [L,R=permanent]

The essence of this is that it says,

"If we got here by using a domain name that is NOT www.domain.tld,
Then redirect to www.domain.tld."

Therefore, anyone entering your site using domain.tld will be redirected to www.domain.tld.

If you prefer to make a non-www domain name your "standard address", then you'd just change it to:

RewriteCond %{HTTP_HOST} !^domain\.tld$
RewriteRule ^(.*)$ [domain.tld...] [L,R=permanent]

In this way, your preserve both of the original "features" - domain name standardization, and upper/lowercase standardization.

No idea if this works with https - probably not unless the .htaccess is in an https-accessible directory - but I have never personally tried it.

Jim

jdMorgan




msg:1521613
 12:47 am on Sep 22, 2002 (gmt 0)

Boule de poils,

If you'd care to post the relevant section of your code, we can look at it. Or, if you prefer, send it to me via WebmasterWorld stickymail.

Jim

ergophobe




msg:1521614
 4:00 am on Sep 22, 2002 (gmt 0)

Once again, Jim completely aces the mod_rewrite quiz. Jim, for understanding mod_rewrite, I find your posts the second best thing after only the manual itself.

Thanks,

Tom

jdMorgan




msg:1521615
 5:33 am on Sep 22, 2002 (gmt 0)

Hey, thanks for the kind remarks guys, but I learned most of it by messing it up! I got to be very good at interpreting my error logs after locking up my server... :o

The important (IMHO) part of my posts is to let people know where some useful and concise documentation is, and right now - since most come here for help with problems they are having right now, and may not want to wait to go buy a book. Anyway, the links are the important part! Especially since just one typo in .htaccess or httpd.conf will likely cause a problem, and possibly a BIG problem.

Modify mod_rewrite directives, upload with ftp, access site with browser, witness disaster, revert server to working backup version, go read error logs, repeat... ;)

Jim

Boule de poils




msg:1521616
 9:36 am on Sep 22, 2002 (gmt 0)

Thanks for your reply!

Here is the .htaccess file (all .htaccess files used on this server are named htaccess.fi) I am currently using:

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^http://(www\.)?domainA\.org/.*$ [NC]
RewriteCond %{HTTP_REFERER} !^http://tempo\.domainB\.org/xiaoping/.*$ [NC]
RewriteRule ^.*\.gif$ http://www.domainA.org/test/stop.gif [R]

The folder to protect is [domainA.org...]

"http://tempo.domainB.org/xiaoping/" is the alternate address for my site, and it works fine (pictures show, both through img src and link).

Here are the addresses you can use for testing (every picture has a link to itself too):

[domainA.org...] --> Should work, but doesn't
[tempo.domainB.org...] --> Works fine, OK.
[splor.domainC.fr...] --> Shouldn't work, OK.

If I remove the second rewrite condition, then the stop.gif picture shows for the second address, which is what is expected. But it is the only case when it works.

I think there's been a small change since last evening, as direct-linking with img src doesn't work anymore with a remote address. I'll contact the server admin to clarify this.

At the root (http://www.domainA.org/), I also have another .htaccess file for 404 errors:

<Files htaccess.fi>
order allow,deny
deny from all
</Files>
ErrorDocument 404 http://www.domainA.org/404.php

[edited by: DaveAtIFG at 4:34 pm (utc) on Sep. 23, 2002]
[edit reason] Changed to "generic" URLs [/edit]

Crazy_Fool




msg:1521617
 10:39 am on Sep 22, 2002 (gmt 0)

there is an easier way to avoid hotlinking without using .htaccess files.

your domain name will normally be mapped to a directory (/web or /htdocs or something like that) within your hosting account, and you will normally place images in a subdirectory of this, ie, /web/images/. you will link to the images with <img src="images/filename.jpg" ...

create an images directory at the same level as your /web or /htdocs directory. change your image links to <img src="../images/filename.jpg .... etc. there is no direct web access to this directory so nobody can hotlink. only your web pages and scripts can access the images directory. voila! no more hotlinking and no .htaccess files!

you should do the same with any sensitive files or information that you want to prevent access to.

DaveAtIFG




msg:1521618
 9:25 pm on Sep 22, 2002 (gmt 0)

Welcome to WebmasterWorld [webmasterworld.com] stapel and Boule_de_poils, we're pleased you decided to join us! :)

Unfortunately, as moderator, it's my job to yell at... err, umm, help you learn the rules... We try to keep discussions very general and to avoid including specific URLs on this board whenever possible. If we don't insist upon this, the board quickly gets overrun with advertising and it's usefulness erodes for all of us. Item 14 in the "Terms of Service" [webmasterworld.com] addresses this in more detail.

I understand how difficult it can be to discuss or analyze mod_rewrite problems without specific URLs but substituting "domain1.com" or "domain2.com" for your actual domain is preferred whenever possible.

WebmasterWorld has a "built-in" email system called StickyMail for those times when an actual URL must be shared. It's available from the Control Panel link at the top of each page.

I dislike editing posts, help me out here, OK? ;) Again, welcome!
Dave

stapel




msg:1521619
 10:20 pm on Sep 22, 2002 (gmt 0)

To "DaveAtIFG":

My apologies; I'd thought Item 14 referred to advertising. Meanwhile, it doesn't appear that I can edit my original post (the "edit" button isn't displaying). Sorry!

Eliz.

Slade




msg:1521620
 11:42 pm on Sep 22, 2002 (gmt 0)

To Everyone:
Please ignore the code I posted above, somehow I transposed it when fixing it up for the no-specific-url clause. As I stated, jdMorgan's code is correct, even his reply to my post!

To Crazy_Fool:
If I wanted to link to the images then, couldn't I do: [domain.tld...] That's basically the same call, and if it's going to be world readable, it's going to be world readable. But then again, I'm thinking that the doubledots (..) should be restricted for security reasons...

jdMorgan




msg:1521621
 4:01 am on Sep 23, 2002 (gmt 0)

DaveAtIFG,
Sorry for my complicity in posting member site URLs - I just focused on the RewriteRules, and ignored that issue! :o

slade,
Ah, you caught that hint about doubledotslashes, too... :)

Boule_de_poils,
Oops, sorry! My posted rewrites were intended to be used in your root directory. You will likely have to add a RewriteBase directive to get them to work. Or, just test them in your root directory, get them working, and then move them to your per-directory .htaccess and test again.

stapel,
One thing that I forgot to mention that may keep mod-rewrite from working is if you are missing the +FollowSymLinks option in your .htaccess (or in the server httpd.conf above it).

Try adding:
Options +FollowSymlinks
Just before the "RewriteEngine on" directive in .htaccess

Jim

Boule de poils




msg:1521622
 6:53 am on Sep 23, 2002 (gmt 0)

Oops, sorry for posting actual addresses, I guess I didn't think it would be advertising. Sorry again! (anyone knows how to edit old posts? The editing link seems to have disappeared on my posts too?)

Anyway, thanks everyone for the help, I'm definitely going to test out the solution you gave!

Crazy_Fool




msg:1521623
 9:55 am on Sep 24, 2002 (gmt 0)

>>If I wanted to link to the images then, couldn't I do:
>>http://www.domain.tld/../images/img.png?

no - it won't work. only pages and scripts on your server can link to directories at the same level as /web - there is no direct URL access to these files. try it yourself and you'll see there is no need for .htaccess files to block hotlinks.

all you need to do is change the directories like this:
from:
/certs
/logs
/users
/web
/web/images

to:
/certs
/images
/logs
/users
/web

then change image links from <img src="images/filename.jpg" to <img src="../images/filename.jpg"

nuff said!

you can also move other sensitive information / scripts / etc out of the /web directory to prevent direct access via the domain name.

richcasto




msg:1521624
 5:32 pm on Sep 27, 2002 (gmt 0)

Any clue why this doesn't work, in my .htaccess in the document root directory? I get a 404 error.

This is on a shared server, so maybe the Apache config is preventing it somehow? (mod_rewrite is enabled, because other rewrites work fine)

RewriteEngine on
RewriteCond %{HTTP_HOST} ^vanity\.domain\.tld$
RewriteRule ^(.*)$ [domain.tld...] [L,R=permanent]

Thanks!

Rich
P.S. Sticky me if you want the real URL's.

jdMorgan




msg:1521625
 5:55 pm on Sep 27, 2002 (gmt 0)

Rich,

Your rewrite rule looks OK...

I assume you have created a subdirectory called "vanity" and that it has pages/files in it that you are trying to access by using a subdomain called "vanity".

Does the subdomain "vanity" exist? In other words, is it set up in your zone file?

Can you look at your error log and see anything wrong with the rewritten request?

Jim

richcasto




msg:1521626
 6:06 pm on Sep 27, 2002 (gmt 0)

Yes, the 'vanity' directory exists.
Yes, vanity.domain.tld points to domain.tld.
I don't know where the error logs are because it's a shared server. My access logs are created nightly for my domains.

jdMorgan




msg:1521627
 1:21 am on Sep 28, 2002 (gmt 0)

Rich,

Contact your hosting company and ask them how to get your error logs. You really need them if you are going to be using mod_rewrite. Even if you are an expert rewrite-rule writer, over time it will save you hours of work trying to find that "one little typo" in .htaccess or httpd.conf that blows up your site.

I don't see anything wrong - Anybody else have any ideas to help Rich out here?

Jim

Key_Master




msg:1521628
 1:44 am on Sep 28, 2002 (gmt 0)

Not every Apache host has mod_rewrite enabled. For those cases, the following should do the job.

SetEnvIf Referer ^$ local
SetEnvIfNoCase Referer ^(http://www\.¦http://)yourdomain\.com local

<Files ~ "\.(gif¦jpg¦png¦css¦js)$">
order allow,deny
allow from env=local
</Files>

Crazy_Fool,

Please explain more, do you have an example to share? It doesn't work for me. I could make your example work if I password protected the directory and used a spider to fetch the images. When you think about it, there is no way for a server to know the image request came from a page on your site (other than the referring URL which is easily spoofed) since it's the visitor's browser that parses the HTML and makes the file requests.

andreasfriedrich




msg:1521629
 2:45 am on Sep 28, 2002 (gmt 0)

>>If I wanted to link to the images then, couldn't I do:
>>http://www.domain.tld/../images/img.png?

no - it won't work. only pages and scripts on your server can link to directories at the same level as /web - there is no direct URL access to these files. try it yourself and you'll see there is no need for .htaccess files to block hotlinks.

Sure enough this produces an 400 Bad Request error. The URI /../images/img.png is invalid.

then change image links from <img src="images/filename.jpg" to <img src="../images/filename.jpg"

Now I don´t understand how that differs from the previous example. An User-agent will need to resolve the the relative URI in the context of the document´s URI the image was referrenced in. This is done according to the following rules:

a) All but the last segment of the base URI's path component is copied to the buffer. In other words, any characters after the last (right-most) slash character, if any, are excluded.

b) The reference's path component is appended to the buffer string.

c) All occurrences of "./", where "." is a complete path segment, are removed from the buffer string.

d) If the buffer string ends with "." as a complete path segment, that "." is removed.

e) All occurrences of "<segment>/../", where <segment> is a complete path segment not equal to "..", are removed from the buffer string. Removal of these path segments is performed iteratively, removing the leftmost matching pattern on each iteration, until no matching pattern remains.

f) If the buffer string ends with "<segment>/..", where <segment> is a complete path segment not equal to "..", that "<segment>/.." is removed.

g) If the resulting buffer string still begins with one or more complete path segments of "..", then the reference is considered to be in error.

RFC 2396 - Uniform Resource Identifiers (URI): Generic Syntax [ietf.org] bottom of [Page 21]

Base URI: [domain.tld...]
Base URI minus last part of path: [domain.tld...]
Base URI plus relative one: [domain.tld...]

Now we need to recursively remove "<segment>/..". Trying, trying, trying real hard, but it won´t work, since "<segment>" is "www.domain.tld" which is not a path element. Now either the UA will fail at that point or it will connect to www.domain.tld and request "/../images/filename.jpg" which is an invalid URI. Sure enough this produces an 400 Bad Request error.

BTW the cause of this error is NOT even remotely related to permissions set in the server config or the file system. There is just no way to name a resource that resides outside of the namespace which you are trying to use to name it.

fiveyes




msg:1521630
 12:59 am on Oct 2, 2002 (gmt 0)

Key_Master's solution is a good one, but only if you're aware of the following restrictions:

[httpd.apache.org...]

(especially the one requiring Apache 1.3.13 or later to add this to a ".htaccess" file at the top of the directory tree you wish to protect, otherwise it has to be used within apache's config file.)

Superman




msg:1521631
 7:02 am on Oct 5, 2002 (gmt 0)

This .htaccess has always worked great for me in stopping hotlinkers:

RewriteEngine On
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+\.)*yourdomain.com/ [NC]
RewriteCond %{HTTP_REFERER} !^http://([a-z0-9-]+\.)*12.345.67.890/ [NC]
RewriteRule ^.*$ [yourdomain.com...] [L,R]

Replace "yourdomain.com" with, you guessed it ... your domain name, and the "12.345.67.890" with your sites IP address.

This .htaccess will protect the contents of whatever directory it is placed in from being hotlinked or linked to. I use it in my "images" directory and my "logs" directory.

-Superman-

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved