I've seen this bot occasionally but the other day it chewed 23 pages or so. Can't find it listed on any of the databases, id please? Thanks.
5:38 pm on Apr 17, 2000 (gmt 0)
And the spiders name/ip address?
6:02 pm on Apr 17, 2000 (gmt 0)
Sorry 'bout that, thought I had listed it in the topic description. Anyhow it's called Crawler.de. FunnelWeb lists it as a robot but doesn't give me the ip. Searching through my raw logs shows no results for crawler.de. I can't seem to trace this one. Thanks.
2:54 am on Apr 19, 2000 (gmt 0)
Crawler.de is a German search engine (and a pretty aggressive one at that). You can check it out at: [crawler.de...]
Here's what we have in our botBase spider wise: #-------- #SE Crawler crawler.de 184.108.40.206 #UA Crawler V 0.2.2 email@example.com crawlit.crawler.de 220.127.116.11 # -------