NutchCrawler

[volatilegx]

IP: 12.236.216.157
UA: "Robot: NutchCrawler, Owner: [e-mail]"

Anybody seen this one or have any info?

[coconutz]

Saw this on 10/11. Same IP and UA. Did a site search that day and it returned one thread. Tried the search again today and no results. Odd.

2002-10-11 18:28:55 12.236.216.157 GET /index.htm 200 Robot:+NutchCrawler,+Owner:[e-mail]

[martin]

12.236.250.97 9 Oct
12.236.216.157 11 Oct

Same UA. The IPs resolve to IP(dashes instead of dots).client.attbi.net

[coconutz]

I sent [e-mail] a message on the 11th and received a reply today. Says that this is an experimental bot that uses wget and does not request robots.txt unless it goes beyond the first page. He said he'd try to correct that.

<added> This is interesting, I just saw this at the bottom of the reply</added>

Winton Davies
Lead Engineer, Overture (NSDQ: OVER)
1820 Gateway Drive, Suite 360
San Mateo, CA 94404
work: (650) xxx-xxxx
cell: (650) xxx-xxxx
[overture.com...]

[coconutz]

Could this be related in any way to the cadabra.com spider [webmasterworld.com]?

[volatilegx]

Thanks guys

[weesnich]

Saw this as well, same IP. It seemed to be interested in the first page of each ODP-listed content.