How to prevent access to files by direct URL entry in browser

[DKDiveDude]

Hello all, this is my first post, and please accept my apology if it has already been covered. If so please provide me with a guide dog to where the treasure can be found.

I have edited the .htaccess file in the public_html folder of my site, so only my domain can access jpg,css,js and txt files, this way:

RewriteEngine on
RewriteCond %{HTTP_REFERER}!^http://mysite.com [NC]
RewriteCond %{HTTP_REFERER}!^http://www.mysite.com [NC]
RewriteRule .*\.(jpg¦css¦js¦txt)$ - [F,NC]

The above works great, however I and anyone else can still access one of these files, by just entering the URL into their browser.

I have tried this other rule but it does not seem to work:

RewriteEngine on
RewriteCond %{HTTP_REFERER} ""
RewriteRule .*\.(jpg¦css¦js¦txt)$ - [F,NC]

What am I doing wrong?

Thanks

[jdMorgan]

DKDiveDude,

Welcome to WebmasterWorld [webmasterworld.com]!

There are two problems, one small and one big. The small problem is that your new RewriteCond using a pattern of "" is not valid syntax. What you are probably trying to do is block a blank referrer, and "blank" is specified in regular expressions using "^$" (without quotes):

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?mysite\.com [NC]
RewriteRule \.(jpg¦css¦js¦txt)$ - [F,NC]

Ref: Introduction to mod_rewrite [webmasterworld.com]

The big problem is that many legitimate users will have referrers suppressed, either because of their browser settings or because their corporate or home security firewall blocks referrers. With my Mozilla Toolbar I can suppress referrers in Mozilla and Netscape, for example. Norton Internet Security, a relatively common program, can be set to block referrers. These settings are easily made and then forgotten. Be prepared to deal with "customer service" issues or loss of business if you block blank referrers.

A better solution would be to use a server-side scripted approach, where the user sees a single script-generated page, which then includes images by calling them by *filename* on the server, not by URL. These images need not even have a URL, and would be accessible only by the script.

Jim

[DKDiveDude]

Hmm, I have already tried what you wrote by adding

RewriteCond %{HTTP_REFERER}!^$

so now it says

RewriteEngine on
RewriteCond %{HTTP_REFERER}!^$
RewriteCond %{HTTP_REFERER}!^http://mysite.com [NC]
RewriteCond %{HTTP_REFERER}!^http://www.mysite.com [NC]
RewriteRule .*\.(jpg¦css¦js¦txt)$ - [F,NC]

And still when I enter a direct URL for a JavaScript file in Internet Explorer or Opera it opens up right away!

What gives?

[jdMorgan]

DKDiveDude,

Try flushing your caches between tests. You are probably seeing a locally-cached copy; If the image is cached, it not requested from the server, and the rewrite cannot be invoked.

Take a look at the other clean-ups I made to your code - they'll make it faster.

Jim

[DKDiveDude]

Thanks for your advice, I have changed my code to optimize it like you suggested.

However I can still open up images and the other file types I specified, by entering the URL directly in any browser, even after I cleared the cache, example in IE - Internet Options¦Temporary Internet Files¦Delete Files¦Delete all offline content

Any other suggestions besides the script, which I know nothing about.

Thanks

[jdMorgan]

DKDiveDude,

Have you tried any "test" rewrites to make sure that mod_rewrite is enabled on your server?

A simple one might be:

RewriteRule ^silly_name\.html$ http://www.yourdomain.com/ [R=301,L]

If this works, you should be able to request "http://www.yourdmain.com/silly_name.html" and see your home page instead.

A few random notes:

1) You may need to add:

Options +FollowSymLinks

ahead of RewriteEngine on

2) Posting on this board removes spaces ahead of "!". Make sure you put them back in as shown in my post above.
3) Similarly, posting on this board changes the solid vertical pipe character to a broken vertical pipe "¦" - make sure you fix those by replacing the broken ones with the solid ones.
4) The rewrite code must be installed in your web root directory, or in a subdirectory which is in the path from your web root to the image directory (You've already said you've got the code in your web root).
5) Your ISP may be caching these images as well as your browser. See Apache [httpd.apache.org] mod_expires and Apache mod_headers to see how to control caching behaviour for your images. (Another approach would be to copy or rename an image on your server, and see if you can access it *the first time* using the new name. If so, then the code isn't working. If not, then it is a caching issue [mnot.net] that is interfering with your testing.)
6) I'm using code that is very similar to what you've got and it works, so there is likely some other issue that is affecting your test results.

Jim

[DKDiveDude]

Thanks for your continuing advice.

There is a thing I obviously do no understand about this code:

RewriteEngine on
RewriteCond %{HTTP_REFERER}!^$
RewriteCond %{HTTP_REFERER}!^http://(www\.)?mysite\.com [NC]
RewriteRule \.(jpg¦css¦js¦txt)$ - [F,NC]

The way I thought is was supposed to be interpreted is:

1) Test RewriteRule first - Is file one of the following...jpg, css, js, txt or twidlydut
2) RewriteCond - If referer is NOT blank
AND
3) RewriteCond - If referer is NOT my website
THEN
4) Exceute RewriteRule - All hell breaks loose!

Written another way:

((If anyone is trying to snatch one of the following file types "jpg¦css¦css¦txt") AND (Referer is NOT blank) AND (Referer is NOT my awesome website)) THEN All Hell Breaks Loose!

But in the above, as I see it, nothing happens if the referer is blank.

[jdMorgan]

DKDiveDude,

Yes, you're right...

What you want is this:

RewriteEngine on
RewriteCond %{HTTP_REFERER} ^$ [OR]
RewriteCond %{HTTP_REFERER} !^http://(www\.)?mysite\.com [NC]
RewriteRule \.(jpg¦css¦js¦txt)$ - [F,NC]

"((If blank Referer) OR (NOT my awesome site)) AND .jpg, css, etc. filetype, then return 403-Forbidden."

That was my error, and it's because I *do not* block blank referers, and I'm too used to seeing it written the way I posted above. Doh!

Again, prepare for customer support issues from blocking blank referers...

Jim

[DKDiveDude]

Ahh, mucho better senor :)

I operate a free adult site that had 7.5 million hits, 172000 unique visitors, 59.8 GB in September, so I need to filter of file snatchers as much as possible, I already have conditions and rules for download programs.

Besides the legitimite users you talked about, firewall that has a blank referer, well maybe they should'nt be on my site anyways :)

Thank you VERY much - that cleared up a lot. I too had seen the first code you showed, hundreds of places, and tried it with no success, now it works.

By the way - can I just add more of my sites/domains, after the second RewriteCond, that is first one OR then several AND conditions?

Example:

RewriteEngine on
RewriteCond %{HTTP_REFERER} ^$ [OR]
RewriteCond %{HTTP_REFERER}!^http://(www\.)?mysite\.com [NC]
RewriteCond %{HTTP_REFERER}!^http://(www\.)?mysecondsite\.com [NC]
RewriteRule \.(jpg¦css¦js¦txt)$ - [F,NC]

Thanks again...

[jdMorgan]

Yes, and you can do "local" ORs inside the RewriteConds themselves for better efficiency:

RewriteCond %{HTTP_REFERER} !^http://(www\.)?(mysite¦mysecondsite)\.com [NC]

Jim

[DKDiveDude]

Thank your for everything, it's been very helpful...