homepage Welcome to WebmasterWorld Guest from 54.196.136.119
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
htaccess and rewrite_mod strange problem
works with one domain and won't work with other
Natashka




msg:1518960
 2:03 am on Nov 22, 2002 (gmt 0)

I have a .htaccess file that 1) forbids hotlinking to my images 2) bans the "bad" UA. It used to work perfectly on my previous host. But recently I had to move my site to another host and suddenly I ran into a problem. The new host gave me the temporarily URL to access my site until my domain propagated (smth like [theirsitename...] My htaccess worked fine. But when my domain name propagated the next day, my htaccess stopped working and started producing 403 page! But oddly enough not all parts of htaccess stopped working, but only the part that involves rewrite_mod. The error pages work fine, and IP ban also works. That's why I think there is smth wrong with rewrite_mod.

And as of today, still the same: with their domain name my htacces works, with my domain name - 403.

I have a feeling that it's my host that have to do something about it, I asked them to solve the problem many times, but they wont fix it (or don't know how?). So I guess I am on my own here :(.

My question is: is there ANYTHING I can do to solve the problem myself? Or is it only my host that has control over it? I would really like to get my htaccess to work. And one more thing, I am not familiar with those technical things, I don't even know if I have a root access here, asking my host will not do any good, they wont answer my e-mails. But as far as I can see, I can access etc/httpd/conf on my server, but the directory is empty, I cannot see httpd.conf file. Probably it means that I don't have a root access....

P.S. Sorry if I posted this in the wrong section.

 

Brett_Tabke




msg:1518961
 2:51 am on Nov 22, 2002 (gmt 0)

Welcome to the board Natashka. Sounds like mod_rewrite isn't turned on. I think you are on the right track there by calling your host.

bird




msg:1518962
 4:25 pm on Nov 24, 2002 (gmt 0)

The 403 doesn't really point to a disabled mod_rewrite. In fact, it seems to be working rather too well... (if it was disabled, you'd probably get a "500 internal server error" instead).

Without seeing the actual rewrite directives, it's hard to say anything more than that. My guess is that you're checking for one domain in the referer, but not for the other. If you can post those few lines from your .htaccess (without the actual domain names), then someone should be able to identify the problem.

jdMorgan




msg:1518963
 1:54 am on Nov 26, 2002 (gmt 0)

Natashka,

Yes, please post, and before andreasf leaves for law school! :)

Jim

Natashka




msg:1518964
 3:00 am on Nov 26, 2002 (gmt 0)

Thanks, bird, for offering help, as it seems like I cannot get any help from my webhost :(

Here is what I have in my htaccess:

-------------------

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Ants [NC,OR]
etc... (here goes a long list of bad UA)
RewriteCond %{HTTP_USER_AGENT} ^ZIP [OR]
RewriteCond %{HTTP_USER_AGENT} Xaldon
RewriteRule /* [mydomain.com...] [R,L]

RewriteEngine on
RewriteCond %{HTTP_REFERER}!^$
RewriteCond %{HTTP_REFERER}!^http://(www\.)?mydomain.com/.*$ [NC]
RewriteRule \.(gif¦jpg)$ [mydomain.com...] [R,L]

ErrorDocument 403 [mydomain.com...]
ErrorDocument 404 [mydomain.com...]

------------------------

The Error documents part works perfectly. BTW, that was my webhost who told me to put URLs in the Error Documents part, and not the relative paths, and that's the only way the Error Documents work.

When I try to access the test folder with this htaccess file via my domain name, I see my 403 page. When I access the same folder via [IP...] (I don't have a dedicated IP), all htaccess works: I see the actual pages, the "bad" UA are really banned, images are really rewritten with nasty.gif, I've tested it.

kamals




msg:1518965
 5:43 am on Dec 2, 2002 (gmt 0)

i have the same problem and i've seen my mod-rewrite is installed in apache. Could someone help me out..

jdMorgan




msg:1518966
 6:11 am on Dec 2, 2002 (gmt 0)

Natashka,

I doubt that you really want to use /* in your first rewrite rule, because it means "any number of slashes, including zero." I would suggest the following changes:

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Ants [NC,OR]
etc... (here goes a long list of bad UA)
RewriteCond %{HTTP_USER_AGENT} ZIP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Xaldon
[b]RewriteRule !^(robots\.txt¦403\.html)$ - [F][/b]

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?mydomain.com/ [NC]
RewriteRule \.(gif¦jpg)$ [b]/nasty.gif [L][/b]

ErrorDocument 403 [b]/errors/403.html[/b]
ErrorDocument 404 [b]/errors/404.html[/b]

You should have only one RewriteEngine on in your .htaccess file.
Do not use a full URL with ErrorDocument - the server will return the wrong code! (302)

You can research the first change in the Apache mod_rewrite documentation [httpd.apache.org]. This is the proper way to return a 403 error code to the requestor. With this change, your custom 403 page will be invoked, but without an intervening 302 redirect. It also allows the 'bots a chance to prove their innocence by fetching robots.txt, and to load the 403.html page you wish to serve to them. Otherwise, if 403.html is forbidden, then when it is requested, it is forbidden, so it will be fetched again in a potentially-endless loop. The modified rule prevents this.

The second change "hides" the url of nasty.gif because it does not update the visitor's address bar. It just replaces any requested image with nasty.gif.

Note: The WebmasterWorld posting software deletes spaces preceding the exclamation point "!" character. It also replaces a solid vertical pipe symbol with a broken vertical pipe "¦" symbol. Both of these changes will need to be undone in any code you copy from WebmasterWorld. Make sure to include a space preceding the "!" in mod_rewrite code, and always replace "¦" with a solid vertical pipe.

HTH,
Jim

Natashka




msg:1518967
 8:12 am on Dec 2, 2002 (gmt 0)

Jim,

I really appreciate your reply, and some things that you say I'll definitely keep in mind (like I will change the Rewrite rule), but just today I finally found a solution to a problem and it appeared to be very easy (kamals, this might help you too, I wonder if we are on the same host) :)

All you have to do is add the following line to the very top of your htaccess file:

Options +FollowSymlinks

That's it, so easy and everything starts working perfectly! I have no idea what the hell does this line mean, but it works.

As for not using full URL in Error documents part, that's absolutely right, I never did it before, but on this particular webserver everything works in a strange manner :) Their support instruct users to put the full URL, otherwise it wont work, I don't know why!

But anyway, thanks so much Jim and good luck to kamals.

Natashka




msg:1518968
 8:39 am on Dec 2, 2002 (gmt 0)

And BTW, what if I want to redirect bad UA to an absolutely different site of mine and nasty.gif is also stored on another site (after all, it's about saving bandwidth on this account), lets say I want to send them all to Tripod.

Then, is it a correct syntax for Rewrite rule?

RewriteRule ^.*$ [account.tripod.com...] [R,L]

RewriteRule \.(gif¦jpg)$ [account.tripod.com...] [L]

It does work that way, but the fact that it works doesn't mean it's correct.

andreasfriedrich




msg:1518969
 3:21 pm on Dec 2, 2002 (gmt 0)

before andreasf leaves for law school!

It seems I had already left. Just saw that today.

Note: The WebmasterWorld posting software deletes spaces preceding the exclamation point "!" character. It also replaces a solid vertical pipe symbol with a broken vertical pipe "¦" symbol. Both of these changes will need to be undone in any code you copy from WebmasterWorld. Make sure to include a space preceding the "!" in mod_rewrite code, and always replace "¦" with a solid vertical pipe.

This is an excellent note that should be present in all threads about mod_rewrite. Sometimes I wrote some warning like that, sometimes I forgot. When I did forget it, it turned out to be a problem for a lot of new users.

Perhaps Brett could make this note available as a style tag. Then all one would need to do is add [ bpn ] for broken pipe notice to one´s post. Another useful tag would be [ wc ]name[ /wc ] which would expand to Welcome to WebmasterWorld [webmasterworld.com] name.

Andreas

kamals




msg:1518970
 4:13 pm on Dec 2, 2002 (gmt 0)

WOW... that line really works... now everything works perfect and thanks a lot to natashka for fixing his and mine. Wat a strange problem man. i appriciate your help guys
thanks again

jdMorgan




msg:1518971
 4:53 pm on Dec 2, 2002 (gmt 0)

Natashka,


As for not using full URL in Error documents part, that's absolutely right, I never did it before, but on this particular webserver everything works in a strange manner happy! Their support instruct users to put the full URL, otherwise it wont work, I don't know why!

Check your server responses with the WebmasterWorld Server Headers [webmasterworld.com] checker. Test by accessing a non-existent and a Forbidden document. Make sure you get the proper 404 and 403 codes in response! In most cases, using a URL in ErrorDocument instead of a path will cause a problem. If it does not work and returns an incorrect response code, complain to your host - Incorrect response codes can cause big trouble with search engine listings, as well as other issues.

I would suggest:

RewriteRule .* http://account.tripod.com/nopasaran.html [R=301,L]
RewriteRule \.(gif¦jpg)$ http://account.tripod.com/nasty.gif [R=301,L]

Jim

Natashka




msg:1518972
 1:09 am on Dec 3, 2002 (gmt 0)

kamals, glad it helped (though I am not a man, I'm a girl LOL ;) )

Jim, thanks, according to that test 403 is fine, but 404 page does really return incorrect response (302), so I'll have to work on it with my host! And my last question is (if you are not tired of me yet :)): what does R=301 mean? I understand it's a "response 301", but why is it there, I mean I never saw the ending like that before. What's the difference between [R,L] and [R=301,L] in this case?

jdMorgan




msg:1518973
 1:42 am on Dec 3, 2002 (gmt 0)

Natashka,

what does R=301 mean? I understand it's a "response 301", but why is it there, I mean I never saw the ending like that before. What's the difference between [R,L] and [R=301,L] in this case?

[R=301] indicates that a 301-Moved Permanently server response should be sent
[R=302] indicates that a 302-Moved Temporarily response is desired
[R=303] indicates that a 303-See Other response should be sent

[R=301] and [R=permanent] are equivalent
[R=302], [R=temp], and [R] are equivalent
[R=303] and [R=seeother] are equivalent

[F] returns 403-Forbidden
[G] returns 410-Resource is Gone
[L] indicates that no further URL rewriting is needed ( this flag is not needed with [F] or [G] )

All this is written up in the Apache mod_rewrite documentation [httpd.apache.org].

HTH,
Jim

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved