grub-client

Can't figure out why .htaccess is blocking.

[bobothecat]

Whoops... title should read "Can't figure out why .htaccess ISN'T blocking" - guess I can't figure out how to change that either :)

Hey folks!

Guess this will be my `official' first post...

I've been trying to block the bot grub-client, as they don't adhere to the request's of robots.txt, unfortunately the entry that I have for them in my .htaccess file doesn't seem to be stopping them.

Here's a copy of what I have in my .htaccess:

RewriteCond %{HTTP_USER_AGENT} ^grub-client [NC,OR]

I've had no problem with banning other bots, but for some reason I just can't get this one `down'.

Any help would be greatly appreciated!

[edited by: bobothecat at 9:00 pm (utc) on July 26, 2002]

[jdMorgan]

bobothecat,

Welcome to WebmasterWorld!

Take a look at your log files, and make sure that the user-agent string actually starts out with
"grub-client" since that's what you've specified by preceding it with "^".

If that's OK, then try escaping the "-" by using ^grub\-client instead.

Also of course, make sure the line above has an [OR] at the end.

If none of this helps, then please post the entire user-agent field from your server logs, so we
can look at it.

HTH,
Jim

[bobothecat]

JD,

Here's a copy from my log file showing the UA... it's pretty much the same each time:

"Mozilla/4.0 (compatible; grub-client-0.3.0; Crawl your own stuff with [grub.org)"...]

In regards to the `line before & the line after':

(before the grub-client)
RewriteCond %{HTTP_USER_AGENT} ^Grafula [NC,OR]

(after)
RewriteCond %{HTTP_USER_AGENT} ^hget [NC,OR]

[jdMorgan]

OK,
So the only problem is that your pattern requires the ua to start with "grub-client" and it doesn't
- it starts with "Mozilla..."

Just leave the "^" off your RewriteCond pattern, and it'll work fine - that means match the pattern
anywhere in the ua string.

^text - match anything that starts with "text"
text$ - match anything that ends with "text"
^text$ - match "text"
text - match anything containing "text" anywhere in the string.

All the above modified by your [NC] which means upper/lowercase doesn't matter.

Cheers!
Jim

[bobothecat]

JD,

Thanks for the info! I'll be giving myself a slap on the forehead with a big `duh' afterwards :)

Guess it's time for me to go out and `help' others.

Your assistance was greatly appreciated :)

[jdMorgan]

Hey, no problem... I learned most of this by screwing it up myself! :)

Jim