Forum Moderators: open

Message Too Old, No Replies

Kavende Iranian Crawler

         

g1smd

8:34 am on Aug 31, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month




Hmm. Two accesses seconds apart. Not seen this one before...


209.190.37.130 - - [nn/Aug/2011:nn:nn:nn +0200] "GET /robots.txt HTTP/1.0" 200 888 "-" "Kavande Crawler 1.0/Nutch-1.4-dev (Iranian National Web Crawler)"


209.190.37.130 - - [nn/Aug/2011:nn:nn:nn +0200] "GET / HTTP/1.0" 410 222 "-" "Kavande Crawler 1.0/Nutch-1.4-dev (Iranian National Web Crawler)"



Nutch gets a 410 by default.

dstiles

7:38 pm on Aug 31, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Don't get many nutches lately. :)

Although I have recently got one with the UA...

Aghaven/Nutch-1.2 (www.aghaven.com)

from colo4dallas on 207.210.234.nnn (207.210.192/18)

That range is blocked, together with the Columbus one at 209.190.0/17

Pfui

9:57 pm on Aug 31, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



207.210.234.227
= mail.visvo.com
= "Visvo (running Aghaven now) | Pick a name, any name..." [webmasterworld.com...]

(I didn't want to hijack the thread so I started a new one instead.)

Pfui

2:33 pm on Sep 25, 2011 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Last night, two hits ~20 minutes apart, each time only asked for robots.txt:

85.25.be.static.xlhost.com
Kavande Crawler 1.0/Nutch-1.4-dev (Iranian National Web Crawler)

That IP -- 209.190.37.133, same 'hood as the OP's -- shows the same UA and this candid one:

MyNutchTest/Nutch-1.4-dev (Testing some algorithms)

209.190.37.128 - 209.190.37.159 => 209.190.37.128/27