Page is a not externally linkable
-- Webmaster General
---- Preventing your site from being crawled
killroy - 5:09 pm on Dec 26, 2005 (gmt 0)Thread source:: http://www.webmasterworld.com/webmaster/10564.htm
set a simple env in http.conf and then only log those without this env set (this is for Apache). This helps em also to clean my logs form images and css/js files.
Brought to you by WebmasterWorld: http://www.webmasterworld.com