want to ban MSNBot & Slurp from ONE dir only

I have the code...will it work?

[walkman]

if I put this under directory to ban (them from), will it work? it works for the entire site.

<Directory /domain/www/offlimits>
AllowOverride AuthConfig FileInfo
Options +FollowSymLinks
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^Yahoo!\Slurp [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Yahoo.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Slurp.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*msnbot.*$
RewriteRule .* - [F,L]
</directory>

will they be able to access the rest of the site? I will ban on robots.txt too, but this is extra, in case they ignore robots.txt.

thanks in advance,

[jd01]

I would make sure I use a .htaccess in the directory I want to ban them from. This way the file is only invoked when a request is made that is within the directory and there is much less chance of banning them from the entire site.

EG /domain/www/offlimits/.htaccess

I believe the correct syntax is:

#RewriteCond %{HTTP_USER_AGENT} ^(.*)Yahoo(.*) [OR]
#RewriteRule (.*) - [F,L]

(No $ on the end of the condition.)

I would also consider [NC,OR] (No Case) and only 1 rule for each engine.

Justin

[arras]

I want to ban SLURP from few directories (only)because my page rocks in Yahoo,if you ask me why i want to ban slurp from the "x" directories the answer is because the directory has pages that Yahoo does not like to be at the top 10 (if you are top and you are not a "friend" after a week you find your 10,000 pages site only the Index indexed in Yahoo. Because [of] that <snip> policy of yahoo i want someone to give me the right robots.txt that will not allow slurp and inktomi bots, because is not only slurp it is inktomi as well that <do this>.

[edited by: jdMorgan at 2:17 pm (utc) on April 12, 2005]
[edit reason] Removed off-topic comments. See TOS. [/edit]