Welcome to WebmasterWorld Guest from 23.20.223.88

Forum Moderators: coopster & jatar k

Message Too Old, No Replies

Best approach for searches within a website

Best solution to for a general search of both static and dynamic content

     
5:10 pm on Mar 2, 2009 (gmt 0)

5+ Year Member



I have been wondering for some time now, what is the best solution for a general search within a website. The options i thought of were:

-Google search within the site (assuming all your pages are indexed in Google)
-Spider your own site using something like PHP Dig. (good for spiering static text and dynamic content, but you need to re-index your site every time there is an update)
-Query the database on dynamic content (But won't find static content)

Andy ideas?

Thanks,
Ryan

8:33 pm on Mar 2, 2009 (gmt 0)

WebmasterWorld Senior Member whoisgregg is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I used to use PHP dig, but for anything decent sized you end up wasting a lot of resources respidering the entire site every day when maybe all that's changed is a page or two.

I use google search and query the database for strings that match the pattern of being a SKU.

Whatever the database turns up I always put in as the top result(s) and blend the design to look like the rest of the SERPs. I then loop through the google results and remove the particular product page(s) if it would also show up there. (Otherwise the user sees duplicate results.)

I also "enhance" the results by looping through the google results and, when possible, adding a small thumbnail of the particular product that the user will find if they click on that URL.

The sites I do this for are particularly well spidered by Google, so it's unusual for a new page to take more than a few days to show up.

2:20 pm on Mar 3, 2009 (gmt 0)

5+ Year Member



Yea that sounds like a good idea. Do you use a specific Google API?
10:30 pm on Mar 4, 2009 (gmt 0)

WebmasterWorld Senior Member whoisgregg is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Well, interestingly I just got an email from Google saying they are changing the whole site search program. I used to use Custom Search Business Edition which was (if I recall correctly) $99 per site, per year. The new program is called Google Site Search and has cost varying based on the number of web pages and the annual search query volume.
12:48 pm on Mar 5, 2009 (gmt 0)

WebmasterWorld Senior Member henry0 is a WebmasterWorld Top Contributor of All Time 10+ Year Member



@whoisgregg

just checked it out
is the "more results..." part of the deal
if so how come do they ask for a fee when they also send your users to competitors!

1:47 pm on Mar 5, 2009 (gmt 0)

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I'm partial to sphinx search. It is lightning fast and you can tweak your index(es) with configuration files as well as your results with the provided APIs.
4:23 pm on Mar 5, 2009 (gmt 0)

5+ Year Member



@coopster

That sphinx search looks pretty cool, but what about for indexing static pages who's content is not stored in a database?

8:23 pm on Mar 5, 2009 (gmt 0)

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



There is also an xml pipe interface, xmlpipe2.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month