I noticed that Looksmart's ZyBorg/1.0 is crawling my site every day. It crawls very deep into the site. Will this keep happening and how much I will profit from it?
Z
jeremy goodrich
7:29 am on Jan 21, 2004 (gmt 0)
LookSmart owns WiseNut, and Grub.org - they have a distributed spider which, because it's a distributed application, may crawl your site a *ton* just to list it *once* in their database.
I've heard rumors of them having traffic in some sectors, but for myself, they always run around 'dead last' for any of the major engines traffic wise.
If bandwidth is an issue, I would give them a 403.
div01
12:55 am on Jan 22, 2004 (gmt 0)
Same here...though I've never gotten any traffic from Wisenut.