- - WebmasterWorld
- -- Webmaster General
- ---- Preventing your site from being crawled
killroy - 5:09 pm on Dec 26, 2005 (gmt 0)
set a simple env in http.conf and then only log those without this env set (this is for Apache). This helps em also to clean my logs form images and css/js files.
Thread source: http://www.webmasterworld.com/webmaster/10564.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com