Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Who owns this spider?



5:06 pm on Apr 17, 2000 (gmt 0)

I've seen this bot occasionally but the other day it chewed 23 pages or so. Can't find it listed on any of the databases, id please? Thanks.


5:38 pm on Apr 17, 2000 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

And the spiders name/ip address?


6:02 pm on Apr 17, 2000 (gmt 0)

Sorry 'bout that, thought I had listed it in the topic description. Anyhow it's called Crawler.de. FunnelWeb lists it as a robot but doesn't give me the ip. Searching through my raw logs shows no results for crawler.de. I can't seem to trace this one. Thanks.


2:54 am on Apr 19, 2000 (gmt 0)

10+ Year Member

Crawler.de is a German search engine
(and a pretty aggressive one at that).
You can check it out at:

Here's what we have in our botBase
spider wise:
#SE Crawler
#UA Crawler V 0.2.2 admin@crawler.de
# -------


Featured Threads

Hot Threads This Week

Hot Threads This Month