Forum Moderators: open

Message Too Old, No Replies

How do i make a search engine?

make search engines?

         

ddsworld

1:24 pm on Aug 25, 2001 (gmt 0)



Does anyone know how to make a search engine with spider and all thet?
Please Reply!

NFFC

2:20 pm on Aug 25, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Start here [xav.com...]

ddsworld

5:44 pm on Aug 25, 2001 (gmt 0)



How do I add a BIG index of webpages to my search engine???

NFFC

5:48 pm on Aug 25, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Start here [xav.com...]

ddsworld

6:30 pm on Aug 25, 2001 (gmt 0)



i did start at [xav.com...] but i only get scripts, and after installing them I see that the databas of webpages is empty :( So i need to fill the databas some how! How?

And how do I get a Folder path form f e g altavista or other search engines? ( /sites/www/.... )

WebGuerrilla

6:37 pm on Aug 25, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member




I don't know of any search engine programs that you can download or purchase that come with a database of sites. (I think that is what you're asking)

IMHO the Fluid Dynamics program is probably the best off-the-shelf program (especially for the price) but you must build the database yourself, and it is not a quick task. If you got it installed properly, you just need to access the admin section, and then start inputing the sites you would like to crawl. Slowly, over time you can build a decent size database.

ddsworld

8:23 pm on Aug 25, 2001 (gmt 0)



OK, thanks

john316

8:50 pm on Aug 25, 2001 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You can dump the entire dmoz database here:

[dmoz.org...]

Brett_Tabke

4:36 pm on Aug 28, 2001 (gmt 0)

WebmasterWorld Administrator 10+ Year Member Top Contributors Of The Month



A) buy a top sun workstation ($50k)
B) buy a commercial license for SQL ($10k).
C) buy addons - (ram, drives, etc) ($15k).
D) write a spider/data miner.
E) write a search db retreival and display system.
F) write "glue" logic for d and e.
G) buy 100gig of bandwidth a month.

That will make you "good" for about 100mil pages at 100k views a day.