Welcome to WebmasterWorld Guest from 18.204.48.199

Forum Moderators: phranque

Message Too Old, No Replies

Website search engine - how to implement?

     
3:27 pm on Aug 27, 2009 (gmt 0)

New User

10+ Year Member

joined:Feb 2, 2009
posts: 8
votes: 0


I am new this part of the game. I have a website that is about to launch and I have one last component to implement, the site search engine and have no idea of where to begin.

I want to find a solution capable of handling 1,000,000+ searches. Any and all help is greatly appreciated. thanks.

4:41 am on Aug 28, 2009 (gmt 0)

Moderator from AU 

WebmasterWorld Administrator anallawalla is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 3, 2003
posts:3773
votes: 21


1M searches over what period?
12:06 pm on Aug 28, 2009 (gmt 0)

New User

10+ Year Member

joined:Feb 2, 2009
posts: 8
votes: 0


great question. per week. thank you.
10:29 am on Aug 29, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Oct 9, 2004
posts:1435
votes: 0


Site search is a complicated problem to solve as there are so many factors that you need to take into account before you get a good search system which specifically addresses the needs of your visitors.

It's often the case that a simple page based search will not do (although there are great free and paid solutions out there if you just need a page based search you will have plenty of choice) - do you need to add other options to the search that makes it easier for people to narrow down the result set (think of how useless a Realty site would be without the options they have for location, price range, house type...).

Once you have decided what features the search should have (which is often good to do before starting coding as the database structure can massively effect the speed of retrieving information depending on your needs) you will need to look at what data that touches on and how resource intensive the searches are. If you can profile the likely mixture of search types you're on your way to getting a full picture of how your system is likely to perform (although you'll never get is spot on, people always do things you don't expect them to).

If you know the amount of work the system needs to do you can then evaluate different systems for handling parts of the search. Some search software delivers better results than others, but that may be at the expense of performance. Fortunately, 1 million searches a week isn't too many to cope with (but it's not a trivial amount) - if you take 'Standard' traffic patterns (yours may be different) you'll find that you'll see around 14,000 searches and hour as the maximum on a Monday around lunchtime - that's an average of 4 a second but you need to have the capability to serve more than that as the amount per second will vary quite a bit and you can have cascading effects of delays when you get too many queries hitting a system at the same time (you may want to quadruple that number to 16 per second to be more confident it should not fall over).

Are you sure about the 1 million a week or is that an estimate? The reason I ask is that building a good site search to 16 queries a second may be expensive (depending on the data being searched).

I know this may have brought up more questions than answers, but that's the nature of site search - it's worth putting a fair bit of planning time into but that often leads to more difficult implementation requirements (the key here is to have a search that people like using, so they come back).

Best of luck with the project.

4:56 pm on Aug 30, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member rocknbil is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Nov 28, 2004
posts:7999
votes: 0


If you don't have the skills or time to grow your own, why not just add Google Search to your site?
12:20 am on Sept 6, 2009 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 20, 2004
posts:2377
votes: 0


I would recommend outsourcing your search to G. It can also be a revenue stream for you if you like.

Using G's Adsense search on our site pays out A very good CPM. I know rates vary, but even at $10 CPM that could mean $40k/month rev share in your pocket.

And a pretty good free search tool on top of that. ;-)