Welcome to WebmasterWorld Guest from 34.231.247.139

Forum Moderators: Ocean10000 & phranque

Message Too Old, No Replies

Blocking the Java UA

Some kind of scumbot (?) requesting non-existent pages

     
1:38 pm on Aug 10, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 9, 2003
posts:1908
votes: 0


I've got some kind of bot requesting a bunch of non-existent pages on my site. From the file names they're looking for, my guess is that they're either looking for content to scrape and steal, or (more likely) looking for particular software installations to try to exploit known vulnerabilities.

Obviously I'd like to block this type of request, but I'm having trouble doing it. They're using a lot of slightly-different UA's, such as the following:

Java/1.5.0_01

The "Java/" part is the same for all of them, but the numbers change frequently. My problem is how to block all such user agents, but NOT block any legitimate traffic that may be using the word "java" in the UA string (I've heard there are some like that).

Essentially I'm lost. What can I add to my .htaccess file to make sure this scumbot, or whatever it is, just doesn't get access to my site anymore?

Thanks,

Matthew

2:12 pm on Aug 10, 2005 (gmt 0)

Senior Member from CA 

WebmasterWorld Senior Member encyclo is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Aug 31, 2003
posts:9074
votes: 6


It could be difficult. For the problematic bot, would it be possible to ban it by IP address instead?
2:47 pm on Aug 10, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 9, 2003
posts:1908
votes: 0


No, it uses all different IP's. It strikes about two or three times per week, always using a different IP address and often a slightly different version number in the UA string.
2:54 pm on Aug 10, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


I've blocked Java and Python by user-agent using this:

# Block Java and Python URLlib except from Google and Yahoo Python
RewriteCond %{HTTP_USER_AGENT} ^(Python[-.]?urllib¦Java/?[1-9]\.[0-9]) [NC]
RewriteCond %{REMOTE_ADDR}!^207\.126\.2(2[4-9]¦3[0-9])\.
RewriteCond %{REMOTE_ADDR}!^216\.239\.(3[2-9]¦[45][0-9]¦6[0-3])\.
RewriteRule .* - [F]

The IP address exceptions allow me to use the Google and Yahoo tools that rely on Java and Python.

I have never seen a "legitimate" Java user-agent since doing so.

Jim

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members