Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- EC2LinkFinder


lucy24 - 12:30 am on Apr 6, 2012 (gmt 0)


It contains options -indexes, which some hosts already have in place and results in a 500 when added to an htaccess.

Huh?! Do you know this by direct personal experience? Saying the same thing over again in consecutive directories should have no effect at all. Especially something as common as +Indexes or -Indexes, which people will use even if they are otherwise deathly afraid of htaccess.

Apache does warn about mixing options with and without plus/minus signs, but even there, all they say is "unexpected results". I do not know whether this term includes, say, a complete server meltdown.

:: peering into crystal ball ::

Cybl00, your redirects are ending up as 500 errors because you made no provision for requests that have already been redirected, leading to a chain of infinite redirects. Matter of fact it should lead to a browser error, unless by "redirect" you mean "rewrite". I went through the same thing for a while until I pinpointed the problem. Same goes for blocking files: make sure they're not blocked from the custom error document, if any.

But really, why are you redirecting unwanted guests to anywhere on your site at all? Just 403 'em at the gate. Redirects are for the borderline cases that might be human so you need to give them a chance to convince you. Some especially pesky visitors may require a redirect to 127.0.0.1 but most of the time a simple lockout is all you can, or should, do.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4427499.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com