homepage Welcome to WebmasterWorld Guest from 54.234.141.47
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
htaccess doesn't stop this agent
i thought this would stop it
incywincy

10+ Year Member



 
Msg#: 89 posted 7:04 am on Jan 3, 2003 (gmt 0)

could any of you htaccess experts help?

i'm getting this agent in my logs and want to stop it.

access_log fragment:
68.97.85.241 - - [02/Jan/2003:20:23:19 +0000] "GET / HTTP/1.1" 200 16353 "-" "Mozilla/4.0 (compatible; grub-client-1.0.6; Crawl your own stuff with [grub.org)"...]
213.51.142.55 - - [03/Jan/2003:05:10:32 +0000] "GET /index.html HTTP/1.1" 200 16360 "-" "Mozilla/4.0 (compatible; grub-client-1.0.6; Crawl your own stuff with [grub.org)"...]

.htaccess fragment:

RewriteCond %{HTTP_USER_AGENT} ^grub [OR]
etc etc

why doesn't this work?

thanks in advance...

 

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 89 posted 7:21 am on Jan 3, 2003 (gmt 0)

RewriteCond %{HTTP_USER_AGENT} ^grub [OR]
etc etc

why doesn't this work?

It doesn't work because the user agent does not begin with "grub", it begins with "Mozilla". The start anchor "^" means the pattern should match anything starting with "grub".

In order to block "Mozilla/4.0 (compatible; grub-client-1.0.6; Crawl your own stuff with [grub.org)"...]

You could use:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.0\ \(compatible\;\ grub [OR]

or, leave the start anchor off, and use:

RewriteCond %{HTTP_USER_AGENT} grub [OR]

Here's a concise Regular Expressions tutorial [etext.lib.virginia.edu] for more info.

HTH,
Jim

incywincy

10+ Year Member



 
Msg#: 89 posted 7:34 am on Jan 3, 2003 (gmt 0)

thankyou very much.

time to brush up on regular expressions!

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 89 posted 7:45 am on Jan 3, 2003 (gmt 0)

You're welcome...

After the brushup on regex start and end anchors, you may want to check through your block list and make sure it's doing what you expect - those stray anchors can mess things up. If you know the user agent starts and/or ends with a particular string, use the anchors - they can speed up the compare measurably. If not, leave them off until you've actually caught a sample, then refine the pattern and add anchors for a more efficient match.

For those who haven't seen it yet, one of our forum hosts, DaveAtIFG, did a bang-up job on this Introduction to mod_rewrite [webmasterworld.com].

Jim

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved