Welcome to WebmasterWorld Guest from 54.163.168.15

Forum Moderators: phranque

Spider Software

   
7:59 pm on Jan 2, 2001 (gmt 0)

10+ Year Member



Does anyone know of any good spider software other than Inktomi's that is out there. Thanks in advance.
2:37 am on Jan 3, 2001 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



fireice68,

Welcome to Webmasterworld in 2001.

>>Does anyone know of any good spider software other than Inktomi's that is out there.

Were you looking for some software to spider various internet sites for you or were looking for a spider which crawls your own site every 24 hours like Inktomi's pay for listing? Once we kow what ou have in mind, I'm srue you will get plenty of help to find what you are looking for.

5:38 pm on Jan 3, 2001 (gmt 0)

10+ Year Member



Software to spider various internet sites.
5:44 pm on Jan 3, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Looking for poems or what?
7:31 pm on Jan 3, 2001 (gmt 0)

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Fireice68 - the problem is we do not know what exactly you are looking for. There are lots of speciality spiders - some are web based and others are desktop apps. There is also licensed technology.

If you are interested in having a *small* search engine, for a type of speciality site check out FDSE [xav.com]. It is a high quality script.

11:00 pm on Jan 3, 2001 (gmt 0)

10+ Year Member



Looking for specifically a web based technology.
11:15 pm on Jan 3, 2001 (gmt 0)

WebmasterWorld Senior Member littleman is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Check out [altavista.com ]
or [alltheweb.com ]
11:21 pm on Jan 3, 2001 (gmt 0)

10+ Year Member



In this, I think I am qualified to help.

[googlebot.com...]

[scrubtheweb.com...]

[excite.com...]

These all are spider based...googlebot works for google, excite.com has architext, and scrubtheweb.com has scrubby, although I've never seen it. Enjoy! :)

Cheers,
Han Solo

PS. Littleman, you mean altavista.com, or av.com, has a spider? Never seen it. ;)

 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month