Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- deny all


Key_Master - 10:10 pm on Feb 27, 2012 (gmt 0)


Is this what your are looking for?

# Deny all
SetEnvIf Remote_Addr .* dont_allow

# Allow some IP ranges
SetEnvIf Remote_Addr ^(?:66\.249\.(?:6[4-9]|[78][0-9]|9[0-5])|74\.125)\. !dont_allow

# Allow global access to certain pages
SetEnvIf Request_URI ^/robots\.txt$ !dont_allow
SetEnvIf Request_URI ^/main-page\.php$ !dont_allow

<Files *>
Order Allow,Deny
Allow from all
Deny from env=dont_allow
</Files>


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4422323.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com