Java/1.5.0_06 Spider Sighting, and Questions

[avendus]

I too have noted the Java/1.5.0_06 spider on my site (discussed in closed thread here [webmasterworld.com...] ).

I'd like to use the code jpMorgan provided:

# Block Java and Python URLlib except from Google and Yahoo
RewriteCond %{HTTP_USER_AGENT} ^(Python[-.]?urllib¦Java/?[1-9]\.[0-9]) [NC]
RewriteCond %{REMOTE_ADDR}!^207\.126\.2(2[4-9]¦3[0-9])\.
RewriteCond %{REMOTE_ADDR}!^216\.239\.(3[2-9]¦[45][0-9]¦6[0-3])\.
RewriteRule .* - [F]

But it generates an Internal Server Error. Any ideas why? I'm only semi-experienced with htaccess, so it's hard for me do diagnose.

Also, on my site the bot is using the IP address 66.215.249.202.

Any suggestions on how to most effectively eliminate this nuisance would be greatly appreciated.

[wilderness]

The forum turns these "thingys[pipe]" "¦" into a broken rather than a solid line.

You'll need to change them all to the solid line on the same key as backslash on most keyboards.

BTW the IP address' are only provided in Jim's rule as exception to allow access from those two ranges, while denying access to ALL other ranges that attempt access under the UA of Java or Python.

[avendus]

Thanks for the tip. After seeing your message, I changed all the half-pipes to pipes, but I'm still getting the server error for some reason. If you have any other pearls of wisdom, please let me know!

[Pfui]

Just to double-check a few things --

1.) Is mod_rewrite available on your server?

2.) Did your .htaccess file work properly -- i.e., no errors -- before you added the example code?

3.) If YES -- do you have the following line preceding all mod_rewrite-related code?

RewriteEngine on

4.) If NO, or if this is your first .htaccess file -- did you create it and upload the file as plain text?

5.) Does the code you used have a space between each {REMOTE_ADDR} and its following exclamation mark? (This forum alters pipes and also lops off spaces before exclamation marks.)

[avendus]

Pfui,

First of all, thank you very much for your generous help. This is an existing htaccess file that uses mod_rewrite without problems.

Adding a space before the exclamation marks solved the problem.

Again, thank you.

[volatilegx]

5.) Does the code you used have a space between each {REMOTE_ADDR} and its following exclamation mark? (This forum alters pipes and also lops off spaces before exclamation marks.)

Good catch, Pfui, I didn't know that. You learn something new every day!

[Pfui]

Happy to help, happy the added space(s) did the trick! But all credit is due Jim Morgan, Modest mod_rewrite Maven. Hardly a jdMorgan [webmasterworld.com] post goes by that isn't instructive -- if only I could get my brain around the details! (...Let alone remember the crucial copy-paste tweaks:)