Forum Moderators: open

Message Too Old, No Replies

Snappy/1.1

urltrends bot

         

Mokita

4:16 am on Jun 14, 2006 (gmt 0)

10+ Year Member



I seem to be seeing a spate of unusual bots this month.

205.138.199.126 - - [14/Jun/2006:04:10:28 +1000] "GET / HTTP/1.1" 200 6867 "http://www.urltrends.com/" "Snappy/1.1 ( [urltrends.com...] )"

Found some interesting info on someone's blog - apparently it doesn't obey robots.txt and in my case, didn't even bother requesting it.

He also said: "... was even fortunate to receive a response by its creator himself, but despite him admitting the bot would show an unintended behaviour that should be fixed, nothing has happened since then. Hence I recommend blocking this bot, because of its rude behaviour and its built-in unsolicited spam feature."

[ow.bbclone.de...]

I've banned the IP range.

bobothecat

7:51 pm on Jun 18, 2006 (gmt 0)



IP seems to hail from a web hosting company...

RewriteCond %{REMOTE_ADDR} ^205\.138\.19[2-9]\. [OR]

Should do the trick... though they do have quite a few other ranges as well. NoZone, Inc., which forwards to steadfast.net ( appears to be reassigned ).

incrediBILL

12:02 am on Jun 19, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Good catch, didn't have this range in my blocked hosting companies list yet.

FYI, the only downside I can see to blocking ranges like this is you probably block employees from hosting companies from accessing your sites as well. Not a good tactic if you sell servers and such ;)

keyplyr

6:04 am on Jun 19, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I've had Snappy disallowed in robots.txt for a year or more; forgotten about it. When I checked www.urltrends.com, they had new caches and data about my site.

Removed listing in robots.txt and am now banning by UA.