homepage Welcome to WebmasterWorld Guest from 23.23.22.200
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Visit PubCon.com
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
htaccess doesn't stop this agent
i thought this would stop it
incywincy




msg:1523687
 7:04 am on Jan 3, 2003 (gmt 0)

could any of you htaccess experts help?

i'm getting this agent in my logs and want to stop it.

access_log fragment:
68.97.85.241 - - [02/Jan/2003:20:23:19 +0000] "GET / HTTP/1.1" 200 16353 "-" "Mozilla/4.0 (compatible; grub-client-1.0.6; Crawl your own stuff with [grub.org)"...]
213.51.142.55 - - [03/Jan/2003:05:10:32 +0000] "GET /index.html HTTP/1.1" 200 16360 "-" "Mozilla/4.0 (compatible; grub-client-1.0.6; Crawl your own stuff with [grub.org)"...]

.htaccess fragment:

RewriteCond %{HTTP_USER_AGENT} ^grub [OR]
etc etc

why doesn't this work?

thanks in advance...

 

jdMorgan




msg:1523688
 7:21 am on Jan 3, 2003 (gmt 0)

RewriteCond %{HTTP_USER_AGENT} ^grub [OR]
etc etc

why doesn't this work?

It doesn't work because the user agent does not begin with "grub", it begins with "Mozilla". The start anchor "^" means the pattern should match anything starting with "grub".

In order to block "Mozilla/4.0 (compatible; grub-client-1.0.6; Crawl your own stuff with [grub.org)"...]

You could use:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0\ \(compatible\;\ grub [OR]

or, leave the start anchor off, and use:

RewriteCond %{HTTP_USER_AGENT} grub [OR]

Here's a concise Regular Expressions tutorial [etext.lib.virginia.edu] for more info.

HTH,
Jim

incywincy




msg:1523689
 7:34 am on Jan 3, 2003 (gmt 0)

thankyou very much.

time to brush up on regular expressions!

jdMorgan




msg:1523690
 7:45 am on Jan 3, 2003 (gmt 0)

You're welcome...

After the brushup on regex start and end anchors, you may want to check through your block list and make sure it's doing what you expect - those stray anchors can mess things up. If you know the user agent starts and/or ends with a particular string, use the anchors - they can speed up the compare measurably. If not, leave them off until you've actually caught a sample, then refine the pattern and add anchors for a more efficient match.

For those who haven't seen it yet, one of our forum hosts, DaveAtIFG, did a bang-up job on this Introduction to mod_rewrite [webmasterworld.com].

Jim

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved