301 redirect returns a 403 - sometimes

redirect site.com to www.site.com plus redirect to new page

[nancyb]

Boy am I confused! Decided to do some maintainence and checking tonight and used SEW's Server Header Checker to check some redirects in my htaccess.

I have site.com redirected to www.site.com - this part works fine.

But, when I type in the url of a 301 redirected page and include the www, the page resolves in the browser but SEW's Header Checker returns a 403..

this is the code in my htaccess

Options +FollowSymLinks
RewriteEngine On
#RewriteCond %{HTTP_HOST} ^site\.com
#RewriteRule (.*) [site.com...] [R=301,L]
RewriteCond %{HTTP_HOST}!^www\.site\.com
RewriteRule (.*) [site.com...] [R=301,L]
some more rewrite conditions for user agents and then file redirects
RedirectPermanent /page.htm [site.com...]

the redirect works correctly in the browser with and without the www and resolves, as expected, to www.site.com/page-new.htm, but the Server Header Checker returns a 403 if I type in www.site.com/page.htm while it returns a 301 if I type in just site.com/page.htm

Hope this explanation is clearer than muddy water.

I can't access SimSpider any more so I'm wondering if this is a quirk with the Server Header Checker or if somthing is wrong in my htaccess.

[jdMorgan]

I don't see anything wrong with what you've posted.

Do you have any rules that limit access to certain IPs, user-agents, etc.? Those might be relevant to your problem.

Are there any "execution order" dependencies in your code? (You need to be aware that directives processed by different modules are processed *by* those modules in the order that the modules are configured. In most cases, mod_alias directives are processed before any mod_rewrite directives are processed. In other words, directives in your .htaccess file are not processed in the order you write them, they are processed in the order that each module finds them, and the order that those modules are executed.)

Jim

[nancyb]

Thanks for the reply Jim.

I do limit access to a number of IPs and UAs, but no other directives. My own IP and UA are not limited though.

I also can't find any 403s in my log files for any redirected page so it seems it is only a problem when I test with SEW's Header Checker by typing in the URL with the "www". However, every once in a while, Googlebot gets a 301 for a "Get" that I haven't been able to find a reason for. Not often, and usually it is a "Get" for a directory without the following "/".

<added> have checked all my code and there is no href for any directory that does not include the final "/" nor was there ever a typo and I can't find any link from another site to the subdirectories without the "/".

[jdMorgan]

The reason I mentioned blocking by UA and IP is that the server header checker does not provide a referrer or a user-agent name. If you block blank referrer -and- user-agent name requests, you may have to make an exception for the server header checker to avoid a 403.

For example:

# BLOCK blank referer -AND- UA except for HEAD (AOL cache uses HEAD)
RewriteCond %{REQUEST_METHOD} !^HEAD$
RewriteCond %{HTTP_REFERER}  ^$
RewriteCond %{HTTP_USER_AGENT} ^$
# Allow WebmasterWorld server header checker by host and IP
RewriteCond %{REMOTE_HOST}  !^westhost.*\.westhost\.net$
RewriteCond %{REMOTE_ADDR}  !^216\.71\.84\.181$
RewriteRule .* - [F]

Jim

[nancyb]

Whew, wipes sweat from brow ...

After reading and reading and more reading of past threads, I came across a post that suggested removing lines from the htaccess until it worked.

I think I finally found what was wrong. I had a RewriteRule!^err403\.htm$ - [F] after a list of "Deny from" statements. At least I hope that was it because when I removed that particular RewriteRule, everything seemed to work correctly and SEWs Checker worked as well as the HTTP Headers Validation on a skin in Opera 7.5 .

Thank you Jim for this and the post you made some time ago about removing lines ;)