Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Who owns this spider?

5:06 pm on Apr 17, 2000 (gmt 0)

10+ Year Member

I've seen this bot occasionally but the other day it chewed 23 pages or so. Can't find it listed on any of the databases, id please? Thanks.
5:38 pm on Apr 17, 2000 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

And the spiders name/ip address?
6:02 pm on Apr 17, 2000 (gmt 0)

10+ Year Member

Sorry 'bout that, thought I had listed it in the topic description. Anyhow it's called Crawler.de. FunnelWeb lists it as a robot but doesn't give me the ip. Searching through my raw logs shows no results for crawler.de. I can't seem to trace this one. Thanks.
2:54 am on Apr 19, 2000 (gmt 0)

10+ Year Member

Crawler.de is a German search engine
(and a pretty aggressive one at that).
You can check it out at:

Here's what we have in our botBase
spider wise:
#SE Crawler
#UA Crawler V 0.2.2 admin@crawler.de
# -------


Featured Threads

Hot Threads This Week

Hot Threads This Month