CobWeb/1.0 (auto classifier)

[GaryK]

CobWeb/1.0 (auto classifier)
82.133.112.nn
bloxx1.gotadsl.co.uk
descr: Nildram IP's (dynamic) For ADSL Accounts

It did not read robots.txt. I can't find any info about it.

[Mokita]

I can't find any info about it.

It seems to be an HTML editor for Macs.

[stonetablesoftware.com...]

At least one person has likened it to FrontPage's ability to download whole sites:
[willmacc.wordpress.com...]

However, it visited a couple of our sites last month (same IP block as you saw) but only took index page from each, and no robots.txt.

I decided against banning it unless it does worse things next time it comes a calling.

Did it take much from your site Gary?

[GaryK]

Nope it didn't take a lot of files. In fact it seems to have done what it did to your sites. Took the default root page for 35 sites and then left.

Thanks for the info about this user agent. Especially the link. :)