Forum Moderators: phranque
I refuse to ever do that just to see some person's website, nor will I change "my" settings to my firewall to be able to see "their" site.
I know this can be done because one webmaster did it before when I brought it to his attention....he did something to his htaccess with one line of code I think it was which made his site viewable to "all" but still maintaining his hotlinking issues safe...
How is this done because I run a link directory and I'm getting tired of telling site owners to fix their configurations.
RewriteCond %{HTTP_REFERER}!^$
in their hotlinking section of their .htaccess file, then it should work.
Jennifer
But it does get frustrating when you send them an email about the broken images and graphics and they think it's up to the end user to make their site 100% functional...
anyways...thanks for the quick reply...we'll see if it works.
It's probably not the firewall specifically that's doing this, it's probably some "privacy protection" software that's installed on your computer. However, blocking a referrer from being sent does little if anything to protect your privacy.
Any firewall that stops the referrer from being sent is overstepping its bounds, and any browser that doesn't send a referrer when it should is broken. If you go to a website that employs this technique, then it absolutely is your responsibility (not the site owner's) to make sure that it displays correctly (in this context) because the problem stems from your system using a faulty method supposedly protecting your privacy.
A site owner who builds a website technically should be available for everyone to see. What is being replied here is the same as saying "can't see my site? Tough Sh*T"
A site should always be built for the end user and not specifically for the site owner him/herself. I hate to quote Spock (star trek) here, but "The good of the many, out weigh the needs of the one"
I am not saying the site owner should throw out their own protection, as I can agree that hotlinking can cost them money from extra bandwidt. But to say its the responsibility of the end user (1000's if not 10's of 1000's of them) to sacrifice their own computer just because a webmaster or site owner refuses to simply make it 100% usable is ludacris.
This can be done...because I've seen it done with a few sites who followed up on it themselves. Think of it this way....if the site in question generates income, think of how much they are losing because many surfers who visit see broken images think its a faulty website and move on and lets others know that the site is screwed.
Note that a URL typed into your browser address bar has no referrer. A search engine spider has no referrer, and many users are behind corporate and ISP caching proxies which block the referrer header.
Access control based on HTTP_REFERER in the client request header is simple, easy, and unreliable.
Note that the code posted above has a problem caused by the "space-remover" function of this board; It should read:
RewriteCond %{HTTP_REFERER} !^$
An alternative that works just as well is:
RewriteCond %{HTTP_REFERER} .
Jim
Just to put some numbers on this, I looked at actual searches conducted (to try to eliminate pretty much all spider traffic) on my website over the past 2 days. Both days the number of users with no referrer was around 7 to 8 percent. To me this is a no-brainer: if you could make a one-line change to your .htaccess file and increase your site traffic by that much, wouldn't you do that in a heartbeat?
This is largely a self-regulating thing. As you say, sites that show broken image icons to visitors probably lose a lot of business.
Jim
<A site owner who builds a website technically should be available for everyone to see. What is being replied here is the same as saying "can't see my site? Tough Sh*T">
There has to be a line drawn somewhere for that. I often use lynx (a text-based browser) or my Treo for browsing. Never do I actually expect the site to look right in these environments. It's my "fault" for using this software, and I take the responsibility of the site not functioning correctly in my browser.
<I am not saying the site owner should throw out their own protection, as I can agree that hotlinking can cost them money from extra bandwidt. But to say its the responsibility of the end user (1000's if not 10's of 1000's of them) to sacrifice their own computer just because a webmaster or site owner refuses to simply make it 100% usable is ludacris.>
This is a flawed statement. Blocking the referrer does not protect your computer, and allowing the referrer to be sent to the web server does not sacrifice your computer. A firewall can do its job just fine and should not interfere with the referrer.
jdMorgan,
<Note that a URL typed into your browser address bar has no referrer.>
Your browser sends no referrer when requesting the HTML page of a URL that you've manually typed in, but the browser sends the referrer of the typed-in URL when it is requesting the images. So a typed-in URL won't "break" displaying images that are using this method.
<A search engine spider has no referrer>
I don't really care if a search engine spider can see my images, as long as it can see my content.
<and many users are behind corporate and ISP caching proxies which block the referrer header.>
I've worked with a lot of firewalls and have not seen one that was configured to block a referrer.
<Added above a RewriteRule (Apache mod_rewrite) that blocks access based on a non-local referrer, either of these lines will bypass that RewriteRule if the referrer is blank.>
This would probably work better. And though someone using software that blocks referrers could technically still see hotlinked images, it's not likely that images would be hotlinked if a large majority of users could not see them, because they would then see a broken image.
jomaxx,
<Yes it's "acceptable" in that it's your site and you can run it however you want. But it seems kind of foolish to go to all the trouble of building and promoting a website, just to block a large and growing number of people from your site for a fairly trivial reason.
Just to put some numbers on this, I looked at actual searches conducted (to try to eliminate pretty much all spider traffic) on my website over the past 2 days. Both days the number of users with no referrer was around 7 to 8 percent.>
Large and growing? Now if I do the same check that you did, I get about the same thing.. less than 10%. But you have to consider that most of these are typed-in URLs. It is not the referrer that your image will use to display itself. You have to look at the referrers of the requests made to the images.. that's where the referrer matters. And this referrer will be sent with a typed-in URL. My statistics show that this is less than 1%. A site with something that would be very tempting to hotlink could decide that losing less than 1% of their traffic to help prevent hot-linking is a no-brainer.
jdMorgan,
<This is largely a self-regulating thing. As you say, sites that show broken image icons to visitors probably lose a lot of business.>
I agree that this is self-regulating, but possibly in the other direction. If enough sites employ this technique, perhaps users will start to see that many sites they visit don't appear to have any images, and they will stop using software that blocks their referrer in the hopes that it somehow protects their computer.
I stand by my statement that on my site, based on the mix of traffic sources I have, the number of users whose browsers show no refering page is 7-8%.
Jennifer
Makes no logical sense I grant ..neither does thinking n*rt*n is a firewall ( don't know which you are using ..just guessing ..it does weird stuff whilst "pretending" to protect ) ...cute GUI ..
Valid reasons for blocking the referrer:
jdMorgan and others have pointed out at least half a dozen very valid reasons why blank referrers shouldn't be blocked by servers in .htaccess, when preventing hotlinking. Check several threads in the Apache forum. I believe in what the experts have said and have acted accordingly.
I return a 1x1 pixel image to the hotlinks, but if the referrer is blank they get the real image. The Net is full of blogs now and without .htaccess I would be in trouble, e.g. due to the links (src) in a big Chinese blog.
However, I can see an increase in browser blocked (=blank) referrers in hotlinked image requests. It looks like some of the Firefox users carelessly block almost everything, because it's so easy. Thus my sites may some day have to block blank referrers as well. I'm not alone, so the OP could be even more annoyed in the future.
Blocking the referrer in a browser/firewall/proxy/cache is sort of a misconfiguration that should be done only after careful consideration and when deemed necessary.
<Organizations may not want referrers to give away information about their secured internal site structure.>
When browsing from an internal company server to the public internet, you simply wouldn't use the same window (or same tab in the case of tabbed browsing) that you used to access the internal company server. I never do.
<Schools may not want referrers to give away identifying information about what school, or sometimes even class, a child is coming from.>
User agents may give this away, but I don't see how a referrer would.
<People running embarrassing searches may not want the sites they visit to know exactly what they were looking for - if they subsequently register or make contact with the site, they could be personally identified.>
Copy and paste the link from the search engine into the address bar :-). I don't know if I'd want software installed the blocks all of my referrers for a few potentially embarrassing searches.
<Also Individuals may not want big marketing companies to be able to track them from site to site with this level of precision.>
Perhaps.. but big marketing companies don't usually have a way to tie this into something personally identifiable. I don't mind if a website that I visits tracks where I, along with 10,000 other visitors, are coming from so they can see what works. I would have a problem if they "attacked" me personally to track me through the entire internet. But this would be done with spyware, not a referrer.
geekay,
<jdMorgan and others have pointed out at least half a dozen very valid reasons why blank referrers shouldn't be blocked by servers in .htaccess, when preventing hotlinking. Check several threads in the Apache forum. I believe in what the experts have said and have acted accordingly.>
Yes, after reading that, I agree that this is a good way to go. If I were to employ hotlink protection, I would think it a good idea to not block blank referrers, just non-local ones.
<Blocking the referrer in a browser/firewall/proxy/cache is sort of a misconfiguration that should be done only after careful consideration and when deemed necessary.>
I would agree that someone could conceivably come up with a reason why it would be a good idea to have all referrers blocked all the time (though I can't think of any at the moment), but those aren't usually the users that actually have referrers blocked. Most of them have it blocked because they've installed software that does this and they are not aware, or because they believe it provides them some level of security or privacy (which really isn't the case).
Re companies: Security does not come from requesting that all your employees always to do things in a certain way; it comes from using a simple software tool that handles the concern automatically and allows you to focus on more important things. Besides, working in a different windows doesn't have anything to do with referrers anyway. I'm talking about following external links from within a secure website.
Re schools: I see this all the time. Kids follow links from a school website when doing research on various topics. The referrer shows the school, and if you look at the page you can often clearly see the which class is working on a particular project.
Re "Copy and paste the link from the search engine into the address bar": What's the point of even suggesting this? Sure there's a possible if impractical workaround for very smart people, but I was simply answering the question why people feel there's a privacy issue in displaying the referrer.
Re marketing companies: Yes, usually not personally identifiable. But again my point was that it's a concern in many people's minds. I wasn't speaking to the issue of whether you specifically mind being tracked.
I love hotlink protection and it's been a huge weight off my mind since I activated it. I really dislike hotlinking, for bandwidth reasons and copyright reasons. I like hotlink protection *a lot*. I cannot emphasize how much I like having hotlinking protection set on all my sites.
I personally would feel no obligation to change my hotlinking settings, even if I got some irate complaints about it. I assume that it's configured correctly (no complaints so far) but even if it happened that a certain percentage of visitors couldn't see the graphics, my feeling is that if I decide to change my htaccess settings, I will do it. But not because I somehow *ought* to do it, but because I would *want* to. And if I didn't want to, then those people who were shut out would just be shut out. They don't pay my hosting fees. I could take my site offline permanently and they'd have no say over that. Why should they have any say over this?
Plus even if the .htaccess was set up carelessly to block blank referrers, at least those surfers would still see a usable site.
The webmaster still has to pay for the bandwidth, but that's not as much of an issue as it was a few years ago. I use a fairly inexpensive dedicated server package that comes with either 800 or 1000 Gb of transfer included. I forget which -- even though I have a busy site I never go beyond about 250 Gb anyway.
<Re companies: Security does not come from requesting that all your employees always to do things in a certain way; it comes from using a simple software tool that handles the concern automatically and allows you to focus on more important things.>
I understand that an administrator needs to have tools in place to secure their network. But security doesn't come from blocking a referrer.
<Besides, working in a different windows doesn't have anything to do with referrers anyway. I'm talking about following external links from within a secure website.>
I would think that having external links pointing from a site that you want to keep secret would be the real problem.
<Re schools: I see this all the time. Kids follow links from a school website when doing research on various topics. The referrer shows the school, and if you look at the page you can often clearly see the which class is working on a particular project.>
You see this all the time? I don't know about anyone else, but this scenario seems like quite a stretch. If someone doesn't want a website to be known about, they shouldn't place outbound links on it. If they don't want others to access it, they should password-protect it. Blocking referrers is just side-stepping the issue.
<Re "Copy and paste the link from the search engine into the address bar": What's the point of even suggesting this? Sure there's a possible if impractical workaround for very smart people, but I was simply answering the question why people feel there's a privacy issue in displaying the referrer.>
So because someone might sometimes want to not know what items they searched for when going to a site, they should have software installed that blocks referrers, effectively breaking the functionality of the browser? It seems much more practical to copy and paste a link when you don't want to the referrer passed versus always blocking it because at some time in the future you might not want it passed once or twice.
<Re marketing companies: Yes, usually not personally identifiable. But again my point was that it's a concern in many people's minds.>
Then people need to be educated so they understand that blocking referrers won't protect their privacy. It's irresponsible to continue to feed that misunderstanding by selling them software and telling them that it's doing something (protecting their privacy) when it's not.
jomaxx,
<Here's a suggestion that just occurred to me... Why not have a parallel directory of images that have your site URL superimposed in small text at the bottom? Then if an image is hotlinked from another domain, you can rewrite the request to point to the second directory. That way the image is still shown but at least you get some free advertising out of it.>
I think this is a very good idea.
