bandwidth leaching prevention for .php images

10% of my bandwidth isn't on my site, but it's on my tab

[Monty]

after looking for several days, I haven't been able to find an answer that actually addresses this particular file type. This is an actual image URL that I would like to only display on my own site and maybe 2 or 3 others.

[examplesite.com...]

What would be greatness would be to have a whitelist and let that serve as where I want the images to work. Behind that would be a blacklist, but that would take longer, since I have to add the sites I don't want, one by one.

I tried several versions of .htaccess files, but in the end, all of them ended up giving a forbidden error to at least some of my forum users, which is running vB. IE, the .htaccess files gave the forbidden error for URL's like [examplesite.com...]

TIA

Mont

[edited by: DaveAtIFG at 9:18 pm (utc) on Aug. 28, 2003]
[edit reason] Made URLs non-specific [/edit]

[robertito62]

Monty, welcome to Webmasterworld.

Don't have the link handy but it will be a good idea to review the TOS. I believe posting URLs is not allowed.

Someone with more expertise on your problem will be posting soon, hopefully.

[Monty]

I apologize for the posting of a real URL, I didn't realize it was not allowed. It's been my experience that when someone can actually see the problem, they have a higher chance of solving it and that's why I posted it. Also, just in case this isn't allowed, I did first inquire in the vB support forums (aka cross posted), which remains unanswered and I am now looking for some outside solution. Google found several relevant threads here, but as far I can tell, none specific to my particular problem.

All of the examples I have seen to solve this problem revolve around .jpg and .gif file extensions, and when I apply them to .php extensions, I catch users trying to enter a forum from either an outside link or bookmark that can't get past the error message. Actually, I would like to block all content sharing with most other sites, since some of them are now taking entire pages with them. But, I don't want to offend my forum users, and so there lies the problem. Again, thanks in advance for any ideas and my apologies for having to be edited.

Mont

[jdMorgan]

Monty,

Pleas post whatever code you got closest to working, and then describe what went wrong when you used it. To comply with TOS, just change the domain names to "mysite", "goodsite", and "badsite.com".

There are many variables involved, and some here may know a bit about mod_rewrite, but little about phpBB... For example, me. :) Also, none of us know anything about how your directory structure is organized.

By filling in whatever bits of information you can, you increase the chance of getting a useful answer.

I assume your blacklist and whitelist will be based upon the HTTP_REFERER. If that is the case, then you must decide what to do with blank referers. Allowing blank referers is necessary if any of your users may be behind corporate firewalls, or using products such as Norton Internet Security. Unfortunately, it also allows user-agents with blank-referers to continue to exploit your images.

Jim

[Monty]

My Dad always says the first mistake is experience, the second time, is being dense, so here's to hoping I am not too dense. :) I changed all the URL's but the translation sites (they have a slightly different syntax) and the one of the actual software. I also fall into a small segment of forums that allow image uploads in the first place, but given my target audience, they would probably choose images over text. Fisherman like posting fishing photos :) My last file is here, however one line I did try in it, isn't. I can round it up, if needed. The problem is getting a forbidden error when going directly to a forum from either a static link or bookmark. Also, several firewalls won't let me through, and I am proud of being allowed into all the local schools and libraries due to clean content.

## Your domain.tld goes here:
SetEnvIfNoCase Referer "^-$" local_ref=1
SetEnvIfNoCase Referer "^http://(www\.)?mydomain\.com(/¦$)" local_ref=1
## any another domains that you want to be able to display your files:
SetEnvIfNoCase Referer "^http://(www\.)?friend1\.com(/¦$)" local_ref=1
SetEnvIfNoCase Referer "^http://(www\.)?friend2\.net(/¦$)" local_ref=1
SetEnvIfNoCase Referer "^http://(www\.)?friend3\.com(/¦$)" local_ref=1
## Allow empty referers (old browsers, private, errors, bookmarks,...):
SetEnvIfNoCase Referer "^$" local_ref=1
## Some translation sites I have noticed:
SetEnvIfNoCase Referer "^http://babel\.domain\.com/" local_ref=1
SetEnvIfNoCase Referer "^http://translate\.domain\.com/" local_ref=1
## To add more, follow the above pattern:
## start with '^http://', change all '.' to '\.', and end with '/'.

## Option #1 file types you want to protect
<FilesMatch "\.(gif¦jpe?g¦png¦php)$">
Order Allow,Deny
Allow from env=local_ref
</FilesMatch>

Just to clarify, I run vBulletin version 2.3.0

[jdMorgan]

Monty,

There are a few minor 'style' errors in there, but nothing that should stop anyone legitimate from seeing your images.

What goes wrong when you use this exact code?

Perhaps I should point out another possible cause of trouble: A lot hinges on the precise defintion of "domains that you want to be able to display your files". The code you posted will allow visitors' browsers to follow <IMG> links on those other domains to see your images. However, it will not allow the servers at those other domains to access your images. The distinction appears minor, but is not. If you wish to allow remote servers to access your images, use "Remote_Host" instead of "Referer" in the appropriate SetEnvIf directives.

Jim

P.S. For more translation services, etc., that you may wish to allow, see this post: [webmasterworld.com...]

[Monty]

Jim,

The current issue/problem is using that exact code, visitors that surf in from a link on an .html page at www.friend.com or from a bookmark on their local machine, directly to a forum, see a forbidden error. They believe that they have been banned. Not so.

I have renamed the .htaccess file for the evening, in order to disable it, and allow most caches to clear. Tomorrow, the plan is to reinstall the file, and begin testing. I read a post here that said that clearing the local pc/isp cache was essential to determine functionality of all this, so that might be part of the solution or my perceived lack of.

[jdMorgan]

Monty,

Good plan to disable it if you're not around...

I'd suggest you simplify this thing until you get it working harmlessly for your users and those of your approved "friend sites." Take out all the fluff concerning translation and image caching, and get the core debugged first.

As I said, I saw nothing really wrong with your code that might cause the problems you're seeing, but here's a trimmed-down version:

## Your domain\.tld goes here:
SetEnvIfNoCase Referer "^http://(www\.)?mydomain\.com(/¦$)" local_ref
## Allow empty referers
SetEnvIf Referer "^$" local_ref
## any another domains that you want to be able to link to your files:
SetEnvIfNoCase Referer "^http://(www\.)?friend1\.com(/¦$)" local_ref
SetEnvIfNoCase Referer "^http://(www\.)?friend2\.net(/¦$)" local_ref
SetEnvIfNoCase Referer "^http://(www\.)?friend3\.com(/¦$)" local_ref
 
## Option #1 file types you want to protect
<FilesMatch "\.(gif¦jpe?g¦png¦php)$">
Order Allow,Deny
Allow from env=local_ref
</FilesMatch>

One caveat: This forum changes the solid vertical pipe character into a broken vertical pipe "¦" character. You must edit these back to the solid vertical pipe character before attempting to use any code copied from this forum.

Jim

[Monty]

Jim, I was told to add this line

SetEnvIfNoCase Referer "^-$" local_ref=1

but wasn't told where it goes. Can you shed some light on the need and placement for it?

[jdMorgan]

It goes nowhere - that is a cute trick to avoid being caught with a blank referer.

Leave it out.

OK, I'll explain... Some sites block blank referers. Apache raw access logs show "-" for a blank referer. So, if a user-agent were to use a referer of a literal minus sign (hyphen) character, then it would bypass any restrictions on blank referers, but appear to be a blank referer in the raw access logs. Some troublemakers also use a hyphen for their user-agent for the same reason.

Since you have opted to allow blank referers, you don't need to worry about fake blank referers. Just leave that line out.

Jim

[Monty]

thanks a million, Jim. I am going to implement the code on Tuesday when I get back from the holiday weekend. Your help is appreciated.