Have you seen this spider before?

vspider

[Rugles]

I do not remember this one before.

User agent = vspider
IP = 203.49.22.165

Does anyone know who this is?
It racked up 3200 hits in less than an hour last night and need to decide wether to ban them.

All help is appreciated.

[digitalghost]

Looks like the Verity Spider, running in persist mode.

It's not very server friendly.

DG

[Rugles]

Would any search engine be using it?

I am not sure I should let them crawl my site again.

[digitalghost]

No, individuals, mining data. I'd exclude it. In persist mode it can hit your site x times per, depending on the variable the user sets.

DG

(must need coffee, hit alt s twice and was still wondering why this message wouldn't post.)

[Brett_Tabke]

Is this it?
[ojp.usdoj.gov...]

[digitalghost]

The Verity Spider default UA is vspider. The IP can be from anywhere. There are crufted up versions of it as well.

The IP given doesn't resolve.

There was a variable in the old version that allowed the indexer to set the dang thing to hit a site n times per. Not sure about other versions. I've seen it in logs from various IPs.

DG