Forum Moderators: open

Message Too Old, No Replies

GrubNG

Is this a Vista crawler?

         

tangor

5:25 am on Feb 19, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Not too many, so to speak, but more every month...

POEGrubCrawler/0.01
GrubNG 20080128
Grub/2.0 (IOI crawler; [index.isc.org...] crawl@index.isc.org)

22 different ips... all over the place.

One asked for robots.txt 3 times on three visits, and nothing else.
Multiple ips as for only three different files

Certainly no abuse here, just curious as to what this is (and if it might be turned to evil at some future time when we least expect it).

jmccormac

5:06 pm on Feb 23, 2009 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Probably the Wikiasearch Grub spider. It is a distributed crawler hence the multiple IPs.

Regards...jmcc