Running a search engine (and spider too)

[rfcon]

Hello everybody. Does anybody have the experience of running a search engine and spider?

I actually run a succesful dental site, and I want to expand the horizons running a very targeted search engine. I've didn't find any dental search engine, so I think that it's a good idea.

I've been testing ASPseek and MnogoSearch and both seems to be the right tools for the job, but they're resource intensive (dedicated server from scratch and 2 Mb bandwidth minimum).

Is worth the investment? Can this be profitable?

Any opinion is truly welcome.

[sugarkane]

The most resource-intensive part is spidering / building the index - if you can do this part 'offline' (ie on your home machine rather than your server) and upload the index to your site, you could probably run those engines quite happily on an ordinary hosting account (for reasonably small amounts of pages at least).

As for profit - I wouldn't be expecting to make much from the engine itself, but if it's integrated well into your site it could be a very worthwhile 'added value' feature that you can use to boost your site's stickyness.

[rfcon]

thanks for your reply sugarkane.

I agree, the spidering/index building is really resource intensive, I was able to fetch 1600 urls in 3 hrs (128 wireless connection).

I know that making profit from a search engine is a difficult task, but maybe a very specific engine has better luck.
Traditionally sites like mine (content) tend to compete, not sharing any piece of information, a search engine could be a good idea for the users, but in terms or profit...

Do you know where can I find some testimonies about running an engine?

[sugarkane]

The good thing about a specialised engine is that your visitors are pre-targetted to your topic area, and hopefully more responsive to relevant advertising and/or affiliate links.

The downside is that the more specialised the engine, the smaller its potential traffic, so it's a trade off really...

You can make a decent profit from such an engine, but it'll take time to build momentum and will probably never make you rich ;)

My overall feeling is to think of it purely in terms of your current visitors: will they benefit from that service, and will it encourage more visits to your already successful site?

If the answer is "yes" then go for it - any extra revenue that the engine brings in through advertising etc may be a very welcome bonus...