Welcome to WebmasterWorld Guest from 220.127.116.11 , register , free tools , login , search , subscribe , help , library , announcements , recent posts , open posts Subscribe to WebmasterWorld
Who owns this spider? joanr msg:396609 5:06 pm on Apr 17, 2000 (gmt 0) I've seen this bot occasionally but the other day it chewed 23 pages or so. Can't find it listed on any of the databases, id please? Thanks.
Brett_Tabke msg:396610 5:38 pm on Apr 17, 2000 (gmt 0)
And the spiders name/ip address?
joanr msg:396611 6:02 pm on Apr 17, 2000 (gmt 0)
Sorry 'bout that, thought I had listed it in the topic description. Anyhow it's called Crawler.de. FunnelWeb lists it as a robot but doesn't give me the ip. Searching through my raw logs shows no results for crawler.de. I can't seem to trace this one. Thanks.
fantomaster msg:396612 2:54 am on Apr 19, 2000 (gmt 0)
Crawler.de is a German search engine (and a pretty aggressive one at that). You can check it out at: [ ...] crawler.de
Here's what we have in our botBase
spider wise: #-------- #SE Crawler crawler.de 18.104.22.168 #UA Crawler V 0.2.2 firstname.lastname@example.org crawlit.crawler.de 22.214.171.124 # -------