| 8:33 pm on Mar 2, 2009 (gmt 0)|
I used to use PHP dig, but for anything decent sized you end up wasting a lot of resources respidering the entire site every day when maybe all that's changed is a page or two.
I use google search and query the database for strings that match the pattern of being a SKU.
Whatever the database turns up I always put in as the top result(s) and blend the design to look like the rest of the SERPs. I then loop through the google results and remove the particular product page(s) if it would also show up there. (Otherwise the user sees duplicate results.)
I also "enhance" the results by looping through the google results and, when possible, adding a small thumbnail of the particular product that the user will find if they click on that URL.
The sites I do this for are particularly well spidered by Google, so it's unusual for a new page to take more than a few days to show up.
| 2:20 pm on Mar 3, 2009 (gmt 0)|
Yea that sounds like a good idea. Do you use a specific Google API?
| 10:30 pm on Mar 4, 2009 (gmt 0)|
Well, interestingly I just got an email from Google saying they are changing the whole site search program. I used to use Custom Search Business Edition which was (if I recall correctly) $99 per site, per year. The new program is called Google Site Search and has cost varying based on the number of web pages and the annual search query volume.
| 12:48 pm on Mar 5, 2009 (gmt 0)|
just checked it out
is the "more results..." part of the deal
if so how come do they ask for a fee when they also send your users to competitors!
| 1:47 pm on Mar 5, 2009 (gmt 0)|
I'm partial to sphinx search. It is lightning fast and you can tweak your index(es) with configuration files as well as your results with the provided APIs.
| 4:23 pm on Mar 5, 2009 (gmt 0)|
That sphinx search looks pretty cool, but what about for indexing static pages who's content is not stored in a database?
| 8:23 pm on Mar 5, 2009 (gmt 0)|
There is also an xml pipe interface, xmlpipe2.