Forum Moderators: open

Message Too Old, No Replies

Please correct this .htaccess exclusion tnx!

         

morales2k

4:19 pm on Jan 11, 2007 (gmt 0)

10+ Year Member



I am trying to exclude the exalead search engine but it is not listening to robots.txt file... so i am trying to kill it or make it see an error page with the htaccess thingy...

so...

for instance... this is the entries i am getting so far...

NG/4.5.1135.28
Mozilla/5.0 (compatible; Konqueror/3.4; Linux) KHTML/3.4.3 (like Gecko)

So... I tried making those agents see an error page but it seems I got it working the wrong way... this are the rewrite conditions i have made up (in bad shape cuz they keep accessing my site)...

# Exabot-Exalead
RewriteCond %{HTTP_REFERER} exalead\.com\/search [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Exabot\/)?[0-9]?.?[0-9] [OR,NC]
RewriteCond %{HTTP_USER_AGENT} ^.*Konqueror.*$ [OR,NC]
RewriteCond %{HTTP_USER_AGENT} ^(NG\/)[0-9]\.[0-9]\.[0-9][0-9][0-9][0-9]\.[0-9][0-9]?.\(compatible[\)\ ] [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Moz+illa\/)?(\(compatible\;\))?(Konqueror\/)[0-9]\.[0-9]\;(Linux\))?(KHTML\/)[0-9]\.[0-9]\.[0-9]?(\(like?Gecko\)) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Exalead).*$ [NC]
RewriteRule .* - [F]

if you know what is it that I am getting wrong, or a good resource to learn to use this utility correctly i will greatly appreciate the help, i know if i use this method of exclusion i will be forever a slave of htaccess and updating it forever... however, for the time being, it is all i can do to temporarily disable unwanted access. thanks.

wilderness

6:19 pm on Jan 11, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



RewriteCond %{HTTP_USER_AGENT} ^(Exalead).*$ [NC]

You have begins with (Exalead)
exactly as typed and including parenthenses.
AND
ends with .*

1) you have failed to escape the period "\."
2) the ends with means your line MUST both begin with and end with exactly as you have specified. NO EXCEPTIONS. (I don't recall seeing [Exalead in parenthenses?]
3) you may eliminate the second line of the multiple lines that you previously provided and in addition add a third bot by using the following for you last RewriteCond line :

RewriteCond %{HTTP_USER_AGENT} ^Exa [NC]

wilderness

12:11 am on Jan 19, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



morales,
Were you able to resolve this issue?