Who are these?

[Hope]

I have been seeing a lot of activity from these guys. Does anyone know who they are?

Dllbot
Excalibur Internet Spider V6.5.4
GenCrawler
GentleSpider
Kenjin Spider
oBot ((compatible;Win32))
xyro_([e-mail])

[littleman]

Hi Hope, I like your nick
We had a similar thread here [webmasterworld.com] - some of the bots on your list are in there.
DIIbot -> [digital-integrity.com ]
"Digital Integrity's patent-pending technology is the solution for
discovering and tracking all types of digital content. The power of
this technology is its ability to detect and track digital "content
segments" -- such as a Microsoft Word document or document
phrase, a PowerPoint slide, a JPEG file, a line of code, HTML, or
even MP3. But unlike a search engine, Digital Integrity's
technology can discover content segments of any length, in any
format."

I don't know what GenCrawler, GentleSpider, Kenjin Spider and oBot ((compatible;Win32)) are - they are probably personal applications. I seem to recall a perl script Bot that was called 'GentleSpider'. If you have the spiders IPs we might be able to do some more tracking.

[Hope]

littleman,

I was really hoping you wouldn't tell me to find the IP. I really hate looking at raw logs. I have heard they can make you go blind. ;) I will take a look and do a search on the IP.

[littleman]

Odds are good oBot ((compatible;Win32)) is some type of desktop app. The thing about User_Agent is that anyone with a bit of skill could hit your site using any UA he/she wants - so the name it self doesn't mean much.

[Hope]

You really know how to make my evening littleman. I was hoping they were all se spiders. It makes me nervous to think someone is spidering our entire site that many times. I wish I knew why. Oh well, guess I should go take a look at the logs to see what they were after and if they took a look at robots.txt.

[Air]

That's a good idea Hope, I'd be interested in what you find.

[Hope]

oBot is very polite. First thing it got was the robots.txt.

The IP for oBot is 195.127.173.165.

The other questionable bots listed above did not look for robots.txt.