Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- deny all


lucy24 - 11:57 am on May 13, 2012 (gmt 0)


RewriteCond %{ENV:dont_allow} ^1$
RewriteRule .* - [F]

But it will only work if mod_setenvif executes before mod_rewrite. On shared hosting, you can never be sure about these things. (It probably does-- I experimented once on my own site-- but it's not iron clad.) If instead you say

Deny from env=dont_allow

you can be absolutely that that sucker isn't getting in.

You can save yourself a little bit of typing, and save the server a few bytes, by using the shortcut BrowserMatch.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4422323.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com