homepage Welcome to WebmasterWorld Guest from 54.146.175.204
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / WebmasterWorld / Webmaster General
Forum Library, Charter, Moderators: phranque

Webmaster General Forum

    
Spider Software
fireice68

10+ Year Member



 
Msg#: 258 posted 7:59 pm on Jan 2, 2001 (gmt 0)

Does anyone know of any good spider software other than Inktomi's that is out there. Thanks in advance.

 

eljefe3

WebmasterWorld Administrator 10+ Year Member



 
Msg#: 258 posted 2:37 am on Jan 3, 2001 (gmt 0)

fireice68,

Welcome to Webmasterworld in 2001.

>>Does anyone know of any good spider software other than Inktomi's that is out there.

Were you looking for some software to spider various internet sites for you or were looking for a spider which crawls your own site every 24 hours like Inktomi's pay for listing? Once we kow what ou have in mind, I'm srue you will get plenty of help to find what you are looking for.

fireice68

10+ Year Member



 
Msg#: 258 posted 5:38 pm on Jan 3, 2001 (gmt 0)

Software to spider various internet sites.

Mike_Mackin

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 258 posted 5:44 pm on Jan 3, 2001 (gmt 0)

Looking for poems or what?

littleman

WebmasterWorld Senior Member littleman us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 258 posted 7:31 pm on Jan 3, 2001 (gmt 0)

Fireice68 - the problem is we do not know what exactly you are looking for. There are lots of speciality spiders - some are web based and others are desktop apps. There is also licensed technology.

If you are interested in having a *small* search engine, for a type of speciality site check out FDSE [xav.com]. It is a high quality script.

fireice68

10+ Year Member



 
Msg#: 258 posted 11:00 pm on Jan 3, 2001 (gmt 0)

Looking for specifically a web based technology.

littleman

WebmasterWorld Senior Member littleman us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 258 posted 11:15 pm on Jan 3, 2001 (gmt 0)

Check out [altavista.com ]
or [alltheweb.com ]

han solo

10+ Year Member



 
Msg#: 258 posted 11:21 pm on Jan 3, 2001 (gmt 0)

In this, I think I am qualified to help.

[googlebot.com...]

[scrubtheweb.com...]

[excite.com...]

These all are spider based...googlebot works for google, excite.com has architext, and scrubtheweb.com has scrubby, although I've never seen it. Enjoy! :)

Cheers,
Han Solo

PS. Littleman, you mean altavista.com, or av.com, has a spider? Never seen it. ;)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Webmaster General
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved