how to create a small search engine!

Meta Or Spider !

[experienced]

Hi,

I am planning to create a search engine may be meta or spider. can anybody tell how should i start and what all things i must consider before doing and which one do you prefer meta or Spider.

Any comment appreciated..

Exp...

[goodroi]

In order to provide adivce people will need to know what your objective is and your current situation.

Do you want to make money?
Do you want to create a community?
Do you want to create a niche engine or a general engine?
What are your resources?

[experienced]

answer below

[edited by: experienced at 5:46 am (utc) on July 23, 2005]

[experienced]

1 - Offcourse i would not mind if my engine will give money to me but my main objective is not to make money as of now. I am only looking for a functionality that can work like a seach engine.
2 - I would like to create a forum where user can come & add their own comments, suggestions, & can post questions & answer.
3 - There would not be any positioning thing in the initial stages but later on we can thing over it.
4 - resources means budget or something else.

apart of this -

Is it possible to display search results belongs any search engine or we need to take permission from the search engine.

Or is it possible to display SERP on our site without permission.

Do engine provide any interface to display there own result on my site.

is it possible to display results 4 search engine resultson a samepage.like 4 listing from each 4 engine.

Do i need any license to running an engine.

Thanks

[experienced]

any help..

[goodroi]

Generally it is necessary to get approval from the search engines since they all have policies prohibiting the use of automated queries. Of course you can just do it and then wait for the search engines to shut you down.

If you are a small player you can use Google's adsense search box, if you are a big player you can negotiate a private deal. If you have alot of internal resources you can build your own index. This is going to be a big undertaking. You will need a good amount of programming time, computers and money.

[Anyango]

i agree with goodroi

but even then if you need to still give it a try, try sourceforge.net, it has many open source search engines built by very senior professioals, you can have a hint and a code to start with

Kami

[Anyango]

and yes you can get results from search engines and show them on your site

Yahoo and Google both provide their search API, you can register and API key for yourself, but according to the liscence that is for personal use and has a certain limit of usage.

Exceeding limit will put a temporary suspension for your queries for that day but using it not personally i mean for somethig else will get you off from em.

[Holmes]

If you're only interested in indexing a few sites you can take advantage of the offer from Gigablast for custom topic search

[gigablast.com...]

I've personally used Teleport Pro which is fine for crawling thousands of sites. If you want an open source solution to index and search I'd recommend Lucene which is part of the Apache product.

I've tended to use Macomedia Coldfusion which includes Verity. But there's a free version out now of NewAtlanta's Blue Dragon, a CFML clone, that integrates Lucene.

[experienced]

Thanks to all of you for your valuable time & suggestions.

First of all i dont have that much high budget with me to purchase a license and also i can not hire a hi fi programmer to build this functionality. What i am planning to do is, i have ready 1 application with me which will show From engine1 Top 10 results, engine2 top 5 and engine3 top 5 results, and any results would not be duplicate.

Now the thing is, if i will start showing these results, would these 3 engine can create some problem now or later on.

2nd - if i will make a tag for these results belongs to which engine like

Search results from engine 1
Search results from engine 2
Search results from engine 3

would this b fine.

Rgds
Exp...

[experienced]

am i wrong ..?