homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Gold Sponsor 2015!
Home / Forums Index / Code, Content, and Presentation / Databases
Forum Library, Charter, Moderator: open

Databases Forum

Need software to build 8m page custom index
ISO large scale index/crawler, recommendations?

WebmasterWorld Senior Member 10+ Year Member

Msg#: 55 posted 2:22 pm on Sep 26, 2005 (gmt 0)

I have looked around at vaious software that can help me transform my metasearch engine into a custom built searchable index of 8 to 10 million pages.

The metasearch option is delivery so-so results and hope this upgrade will deliver supreme style results.

Unfortunately, haven't found one that can handle what I need to do. All too small or require too much hardware.

If anyone can recommend a pre-made sw package that can crawl/index of 8 million pages and deliver a customized search results, please contact me as soon as possible. I know the SW exists, just can't get a grip on what can do the job as need. I checked with siets.net, they came close, but too much hardware requirements were needed.

---Added, how hard/how much would it cost to pay a programmer to build a this program that can do such a thing? Where would I look to find one? Is it possible to for a perl man?

My whole goal is to improve the serps beyond what a metasearch engine can do. I want to improve serp results for my users - bottom line necessity if I am going to dominate my competitors.

Thanks you so much for your help,

Kind Regards,



R e b r a n d t

Msg#: 55 posted 7:36 am on Sep 28, 2005 (gmt 0)

Looking for SE/crawlers/indexers in Database forum will not get you many results (-;

Global Options:
 top home search open messages active posts  

Home / Forums Index / Code, Content, and Presentation / Databases
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved