Welcome to WebmasterWorld Guest from 126.96.36.199 , register , free tools , login , search , pro membership , help , library , announcements , recent posts , open posts Become a Pro Member
Spider Software fireice68
Does anyone know of any good spider software other than Inktomi's that is out there. Thanks in advance.
Welcome to Webmasterworld in 2001.
>>Does anyone know of any good spider software other than Inktomi's that is out there.
Were you looking for some software to spider various internet sites for you or were looking for a spider which crawls your own site every 24 hours like Inktomi's pay for listing? Once we kow what ou have in mind, I'm srue you will get plenty of help to find what you are looking for.
Software to spider various internet sites. Mike_Mackin
Looking for poems or what? littleman
Fireice68 - the problem is we do not know what exactly you are looking for. There are lots of speciality spiders - some are web based and others are desktop apps. There is also licensed technology.
If you are interested in having a *small* search engine, for a type of speciality site check out
FDSE [ xav.com]. It is a high quality script. fireice68
Looking for specifically a web based technology. littleman
Check out [ ] altavista.com or [ ] alltheweb.com han solo
In this, I think I am qualified to help.
These all are spider based...googlebot works for google, excite.com has architext, and scrubtheweb.com has scrubby, although I've never seen it. Enjoy! :)
PS. Littleman, you mean altavista.com, or av.com, has a spider? Never seen it. ;)