[Help] Block by Referer

How to use .htaccess

[mlino]

Hi Guys!

I'm tring to use mod_rewrite to allow only domains owned by me to access some home pages in my server, preventing others site to use some of my resources.

I try to use the following rewrite rules...


RewriteEngine on

RewriteCond %{HTTP_HOST} !.*?atcbsb.com.br$ [NC,OR]
RewriteCond %{HTTP_HOST} !.*?atcbsb.com$ [NC,OR]
RewriteCond %{HTTP_HOST} !.*?virtualradar.com.br$ [NC]

RewriteRule ^(.*)$ [atcbsb.com.br...] [R,L]

But seems it's not working...

Cheers,

Mlino

[wilderness]

Please use example.com in submissions.

1) doing domain name lookups for each request is CPU intensive and may bite you in the backside very quickly.

2) for Condition lines use RewriteCond %{HTTP_REFERER}
a) Please note that using the refer will not provide uniform results, as many request do not include referrals.

3) best practice is a simple denial of access. Redirecting to another page or another website only seems to motivate further annoyance.

FWIW, hopefully another will be along to provide the perils of rewriting to index.php

[lucy24]

to access some home pages in my server

Where are the "some home pages"? The Rule as written would seem to grab all incoming requests-- and if that's what you're doing, how will anyone be able to reach your web page?

another will be along to provide the perils of rewriting to index.php

To say nothing of initial non-captured .*

I think he's busy next door celebrating his birthday ;)

[mlino]

Let me try to explain the situation...

I have three domains and each one has it's own server in different places.

In one of them, let's call it server1, I have a page e.g. player.html. I would like to able to block all requests for this page if they don't come from one of my servers.

Thanks,

MLino

[lucy24]

Ah, so you really do want to keep everyone from seeing a particular page :)

First step is to constrain the rule to requests for that page-- assuming you've got other pages on the domain that are publicly accessible. Put that part in the rule:

RewriteRule ^specialsecretpage\.php$ - [F]

Then you backtrack and put in conditions.

Your big goof is in using {HTTP_HOST}. That's the domain you're asking for. You want {REMOTE_HOST}, meaning the one doing the asking. Give the name in its exact canonical form, like

!^www\.example\.br$

No need to mess about with [NC] or ()? Your domain names are going to be in the right form already.

Now, if your limited-access pages were in a directory by themselves you could do the password-protected 401 route instead.

Final assumption is that you really are talking about requests coming from the domains themselves. If instead you're talking about requests referred by your own domains, then you go to {HTTP_REFERER} as wilderness said. In fact at that point your code becomes almost identical to an ordinary no-hotlinks routine, except that you're blocking pages instead of pictures. And then you need to worry about browsers that don't send a referer, and other forms of "cold" entry.

[mlino]

Hi Lucy24!

Thanks for your replay ;-)... Actually I want to prevent hotlinks from my page, like you mentioned with something like no-hotlink routine used for images.

I adjusted my .htaccess to the following:

Options All -Indexes

RewriteEngine on

RewriteCond %{REMOTE_HOST} !^www\.atcbsb\.com\.br$ [OR]
RewriteCond %{REMOTE_HOST} !^www\.atcbsb\.com$ [OR]
RewriteCond %{REMOTE_HOST} !^www\.virtualradar\.com\.br$
RewriteRule ^flashplayer\.html$ - [F]

Unfortunatelly, nothing change... I can still access the page from a hotlink in another site. Any other tip or trick to make this happens?

Cheers,

MLino

[g1smd]

Clear your browser cache before each test.

[lucy24]

Uhm...

RewriteCond %{REMOTE_HOST} !^www\.atcbsb\.com\.br$ [OR]
RewriteCond %{REMOTE_HOST} !^www\.atcbsb\.com$ [OR]

... et cetera

OK, so I had to rewrite this post three times before I figured it out. Once you do clear the cache, wouldn't this mean that nobody gets in?

[mlino]

Hi,

I cleared the cache and done some tests using Safari, Firefox and Chrome to assure that the solution was working and blocking requests comming from hosts not included in .htaccess file.

I was satisfied with the results, cause it seemed to be working as expected, until I have made a new test using a computer connected to other DSL.

On my site I have a link for the page I would like to block, and when I tried to open that page using this new computer and with a new IP address, all requests comming from my domain names included in .htacces file were blocked also... :-(

It Seems that is working only with requests made by my own server IP address and refusing all other requests outside my network.

MLino

[mlino]

Hi Lucy24,

If you want to see it in action, please go to www.atcbsb.com.br and choose the option <ESCUTA ATC> at main menu, and then <PLAYER FLASH>.

For me the request for the page is being forbidden, even comming from my own site.

Cheers,

MLino

[Leosghost]

It works as you wish..403 ( wimpy/****** )to me in France..

[g1smd]

Lucy gave the biggest clue...

"NOT this OR NOT that" will fail for ALL requests, because one of them will always be untrue.

You need "AND" here.

[mlino]

Hi Leosghost,

I wanted just the opposite. Allow visitors to my site to load that page and block requests for the page from other servers.

There is one site that loads my html (flashplayer.html) page in one of their iframe in his onw home page without my permission.

I know it's more easy to ask them to stop loading my page as they do now, but I don't know how many others sites are doing the same. So I need to block this practice.

Thanks!

MLino

[Leosghost]

Ah my mistake :) I read the thread whilst doing about 6 other things..

[mlino]

Hi g1smd,

As far as I understood reading the mod_rewrite docs, [AND] is implicit. I removed all [OR] flags and still the same.

Now I have the following:

Options All -Indexes

RewriteEngine on

RewriteCond %{REMOTE_HOST} !^www\.atcbsb\.com\.br$
RewriteCond %{REMOTE_HOST} !^www\.atcbsb\.com$
RewriteCond %{REMOTE_HOST} !^www\.virtualradar\.com\.br$
RewriteRule ^flashplayer\.html$ - [F]

Thanks,

Marcio

[g1smd]

'If it is none of the above, then block access' is what it should now do - or it would if you were testing REFERRER.

Clear cache before testing.

[mlino]

g1smd,

It was supposed to work, but it didn't work, even clearing the cache before try. Could you try yourself by accessing the address www.atcbsb.com.br and choose the option <ESCUTA ATC> at main menu, and then <PLAYER FLASH>. and tell me if you get the 403 message?

Thanks,

Marcio

[Leosghost]

It still gives 403

[lucy24]

And, ahem, you've got to stop pointing people at your specific site, because a Moderator will be along any minute to cut out the references.

It still gives 403

As written, that's what it is supposed to do. If you are anyone other than the OP, requests will be blocked. If you want to block by referer you have to change {REMOTE_HOST} to {HTTP_REFERER}.

And then, as I said somewhere up the line*, you have to deal with the ones that don't send a referer. Most will be robots, but some normal human browsers do the same. Will they have an alternative route, or simply "Sorry, folks, you're out of luck"?


* Or possibly: as I started to say and then edited. As I meant to say, at least.

Oh, btw, "Options All -Indexes" is not an optimal wording. There's a place on Apache that explains it in detail. The short version is that you can simply leave out the word "All". If you need to enable options that were turned off at some earlier/higher stage, list them by name with leading + sign.

[Leosghost]

As written, that's what it is supposed to do. If you are anyone other than the OP, requests will be blocked. If you want to block by referer you have to change {REMOTE_HOST} to {HTTP_REFERER}.

This I knew ;-))..but I had got the impression from Marcio's post immediately before mine, that for him it was not showing a 403 ( even from a IP other than his own) and that he was worried that others would not see one :)..

But I still don't think that is what he wants ?
IIUC he wants me and anyone else to be able to see the page(s) when accessing from his main site, via the link..but that the other site(s) using his content via "iframing" the file in his player should be blocked from doing so..?..

[mlino]

Leosghost,

Thatīs exactly what I want... Allow visitors seeing the page when accessing from my main site and prevent hotlinks into their own iframes.

Any other idea?

Thanks,

MLino

[g1smd]

Are you sure you know the differences between:

HTTP_HOST

REMOTE_HOST

and

HTTP_REFERER

and when to use each one?

[mlino]

g1smd,

No. I not sure about the differences between HTTP_HOST and REMOTE_HOST, but I know about the HTTP_REFERER.

Could you can explain a litte about those variables?

Cheers,

MLino

[lucy24]

but I know about the HTTP_REFERER

Are you sure you know? We're not saying this to be mean. But keeping in mind the stuff about "the things you do know that ain't so" et cetera. (I can get someone to translate that into Portuguese if you are bewildered. Really.) See, you talk about referers, but you keep coming back with code that says _HOST.

HTTP_HOST: The site where all your stuff is happening. The site where the htaccess or config file is located. Call it mysite.com (moderators, please look away).

REMOTE_HOST: The visitor's own location. For humans, this should resolve to an ISP, or an institution such as a university or big corporation. Call it yoursite.com (mods, continue looking elsewhere).

HTTP_REFERER: The link or site that sent the visitor to your page, or that asked for follow-up files. Call it othersite.com.

When the requested file is an image or similar, the HTTP_REFERER is normally your own page asking for its pictures and stylesheets. If it's someone else's page, that's a hotlink. Note that the server can't tell if the image is automatically displayed on the page, or if someone clicked on a link to bring up the image.

When the requested file is a page, the referer might be either another of your own pages, or some other page on someone else's site. (See approximately 87,000 threads over in the Link Development and Google sections of this forum.)

So to a human, "referer" is two entirely different things. It can be a page asking for its own images, OR it can be a link that you clicked to go to a new page. But to the server they are exactly the same. All it hears is the browser ("User Agent") saying "such-and-such site told me to ask for such-and-such file".

[Kendo]

Allow visitors seeing the page when accessing from my main site

Yes, use REFERER. Then if the referer is not your site you can redirect to a page explaining that you prefer not to allow access from bookmarks and hotlinks. Note that I mentioned "bookmarks" because it may be a return visitor with honorable intentions, in which you only use such tactics for security reasons.

[mlino]

Lucy24,

Thanks for your explanation. I think you don't need to bother with translate your post to portuguese.

Cheers,

MLino

[mlino]

Hi Kendo!

I will try another approach to this issue, maybe using PHP session.

Thanks,

Marcio