homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / WebmasterWorld / Ecommerce
Forum Library, Charter, Moderators: buckworks

Ecommerce Forum

This 51 message thread spans 2 pages: < < 51 ( 1 [2]     
Site Search - Weakest function of most commerce sites
And never discussed here before

 9:37 pm on Aug 13, 2006 (gmt 0)

As commerce sites grow in complexity and product offerings, the shopper now faces the arduous task of finding the product he wants, even when the site offers it. Years ago we offered only 20 best-selling products and grouped those in a few simple categories. That worked well.

Now we offer far more products and some don't fit into neat product groupings. We've been forced, like most sites, to rely on a "google-like" Search Form that allows customers to enter their own keywords.

Sounds great, until you analyze what customers look for and what they find... Often NOTHING

Last month we installed software that reveals how poorly our search process works. Furthermore, running the same searches on competitors' sites showed their search forms were often worse than ours.

Product searches rely on words found in product descriptions, product names and key words the site owner supplies. The trick to getting the keywords right is to think like a customer (or 1,000,000 of them!).

A sporting goods site may sell "baseballs," but in most cases a search for "base balls" or "baseball" returns nothing. Certainly "base-balls" or "base bals" will come up with zip.

You think the customer will modify his search and try again? Our research indicates they rarely do that. One search and often they're gone

I can't get over how stupid some customers can be in searching. Many searches contain obvious typos or misspellings. Incredibly, one customer insisted on running together multiple words: "needtobuywidets."

Site owners and webmasters fail in the keywords they supply. Shoppers tend to search in plurals, [widgets], yet many searches only turn up the product in singular. That they fail isn't surprising. Getting the keywords right is hard work and it has to be polished thru months of research that few sites conduct. Small sites are utterly clueless, of course.

What's the answer? First avoid free form searches if possible. At best they're inadequate. At worst, they're a site killer. Second, realize their limitations and provide search by categories, too. Obtain software to see what is being searched for, and polish and repolish the keywords your site presents.

Do you have any tips to share in helping visitors find offerings?



 11:36 am on Aug 16, 2006 (gmt 0)

Here is a simple solution that we use for our site It has the free Fluid Dynamics Search Engine onboard for text search (same as WebmasterWorld used to have before the site got too big). Periodically, we download the log and open it in Excel, sorting keywords/phrases in ascending order according to the number of search replies they received. We then delete all with more than 0 search replies.

The resulting list of unsuccessful searches is then processed manually and the keywords or phrases added to the Keywords META-tag on the pages we feel ought to have shown up. After that we just regenerate the database. FDSE indexes the contents of Keyword META tags, so this takes care of the problem. The job takes a day at the most every four months.

[edited by: lorax at 3:17 pm (utc) on Aug. 16, 2006]
[edit reason]
[1][edit reason] no URLs please [/edit]


 12:55 pm on Aug 16, 2006 (gmt 0)

On an ecomm site, to help out with the too many results problem, you may want to try sorting the search results by sales volume of item.

That way, based on probability, you have a greater chance that the item they are searching for is on the first page or two.


 1:19 pm on Aug 16, 2006 (gmt 0)

rencke, I think your site would benefit more from a better laid out top navigation bar. Currently, it really blends in and is a bit hard to read/find.

I demo'd the google mini search appliance for one of our sites, and will be purchasing it without hesitation. For only $2000 you can serach 50,000 documents, have related searches, misspellings, and a powerful search engine, all without getting into much coding.

Search is too important to do poorly.



 3:21 pm on Aug 16, 2006 (gmt 0)

We need a search that returns a product thumbnail. That's another reason most commerce site searches work poorly. For many items, especially apparel, a picture really is worth a thousand words.

Can Google do that?


 10:51 pm on Aug 16, 2006 (gmt 0)

I added a Levenshtein Distance to make my search a little smarter. Really helps with mispellings or multi spellings.

Things like location searches as well.

So searching for "frt mcMuray" would return

Fort MacMurray
FT MacMurray
Fort McMurray
Ft McMurray

You can set the sensitivity to your liking. Another thing I do is catalog all content from all my pages and keep them in an index, any changes to my content updates the catalog and the index. When I do look ups for searching I search my catalog rather then content. Much faster. This along with Levenshtein makes my sites so easy to search. Sorry for the bragging but I really love it.

If you are not familiar with Levenshtein you can have a read about it here.


It is available in many languages, I think even PHP has a Levenshtein call. Usally comes in some type of soundex library.

[edited by: lorax at 12:00 pm (utc) on Aug. 17, 2006]
[edit reason]
[1][edit reason] no URLs please [/edit]


 11:47 pm on Aug 16, 2006 (gmt 0)

Levenshtein Distance

You have reason to brag -- very savvy. Thanks for the link and the terminology.


 2:34 am on Aug 17, 2006 (gmt 0)

jsinger, you can setup the search on the google mini to return XML, which then you can present in any way you want. To get additional info to display, you'd probably have to code in lookups for each item returned.

For an ecomm site, I'm not so sure about google being the best answer.


 9:41 am on Aug 17, 2006 (gmt 0)

Super thread.

I've been trialing Picosearch for a couple of days now and I'm pretty impressed, very configurable and worth a look if your looking for a hosted solution.

Oliver Henniges

 10:31 am on Aug 17, 2006 (gmt 0)

> I think even PHP has a Levenshtein call

As I said, yes it does. I must admit, I read the manual too quickly. Seems indeed very helpful.

Let me raise a related question:
How would you handle spaces in search queries?

If someone was searching for "red widgets" and your product desription does comprise both words but in a different order with some other words in between, the levenshtein function wouldn't help.

I guess in such cases you need a very distictive algorithm: First you perform a search as if the two/three/four-word-search phrase was put in quotes. If this returns zero results, you check whether there are spaces. If there are none, you use the levenshtein function.

Else you do a second search as if all single words were connected with a logical "and". Thirdly you perform a search as if all were connected with a logical "or", and only if this reveals zero, you use the levenshtein function over each single word.

Indeed this is a very interesting topic, particularly because an elaboration of my much too quickly self-designed products-search-routine is quite on top of my agenda. Seems to me now that it is possible to write such a little program with relatively little effort.

Oliver Henniges

 10:42 am on Aug 17, 2006 (gmt 0)

> Picosearch

I used to use picosearch before I began to program in php. I doubt it will allow to perform data-base-driven searches, because for instance my hoster does not allow access to the database from outside for security reasons.

So you'd have to provide idempotent urls for all your product-ages in order to make these accessible for picosearch. You'd also have to reindex your website after every change in your database. This seems only helpful for smaller sites.

What about performance issues? Using levenshtein under php you'd have to read all your product descriptions and loop thru them with the keyword. Are there any means to embed levenshtein into a mysql-search-query?


 3:04 pm on Aug 17, 2006 (gmt 0)

Oliver Henniges, I will address your question about multi words with spaces.

What I do is split the string on ' ' and then create a list, loop through the list of words and find results for each term individually. I also added in some extra logic that will look for the '-' (minus sign) and then take the following word and say 'term not in'.

Oliver Henniges

 3:15 pm on Aug 17, 2006 (gmt 0)

> loop through the list of words and find results for each term individually

yes, that would be the logical or-operation, which in many cases might possibly privide quite a lot of results. Before that it might be convient to perform something like

select * from products where splitpart1 like '%wholetext%' and splitpart2 like '%wholetext%'

and only if this also gives zero results one might search for every single word.

[edited by: Oliver_Henniges at 3:15 pm (utc) on Aug. 17, 2006]


 3:33 pm on Aug 17, 2006 (gmt 0)

I've done quite a bunch of research on this, when i designed a former employer of mine's search results usability and accuracy.

Here are some of the sites i refer to, for good info on search results and design...

Effective Search Results by Kristoffer Bohmann, October 10, 2000

Nielsen Norman Group Report E-commerce User Experience: Design Guidelines for Search

If you have the money, go by the report by Jakob Nielsen, it's incredibly smart and practical...

Also now a free report, check out this pdf from 37 Designs

37Signals: Search Usability Research Report

Good luck...

And yes the biggest problem is how people search, they don't spell well...

But what you can do is take the info of results that either give zero records or no profits and create a translation table.

So if they say bibel bok, you can translate that too bible book, and then change the keyword into the search results, and whammo the end user gets better results...

It can be done, just be practical and smart.

[edited by: lorax at 4:24 pm (utc) on Aug. 17, 2006]
[edit reason]
[1][edit reason] removed urls [/edit]


 4:13 pm on Aug 17, 2006 (gmt 0)

I normally buy from paper catalogues. I sit on the sofa with a coffee and make my choices, then go to my PC to place my order.

If I have a catalogue I want to go straight to that product. Easy enough as each item normally has a code for putting on the paper order form. Oh no! I now have to do a site search which doesn't register the descriptive terms in the catalogue or guess the category, which seldom fits the layout of the catalogue.

PLEASE, if customers get advised of products in printed literature do have a short cut direct to the product on the site.


 5:24 pm on Aug 17, 2006 (gmt 0)

piatkow.... that is an excellent user view point/use case.

One of my bigger clients shop site's does allow you to enter a catalog number to find something you saw in a catalog.

One other thing we just did last year that expands on this, coupons for certain products, if there was a series of coupons put out in pdfs or catalogs for 1 specific product then we would let you enter your coupon number and it would take you right to the product detail page for the discounted item. We recieved a lot of good feedback over this.

Oliver Henniges

 5:50 pm on Aug 17, 2006 (gmt 0)

> piatkow.... that is an excellent user view point/use case.

Yes, indeed. Quite recently I have begun to import hundreds of products from my suppliers catalogues into my database in order to grab that long tail and deliver these products on demand. We also send these catalogues by post or offer one for download.

It is quite amusing: People have become so used to product ids and customer ids meanwhile, quite often we receive fax-orders covering only the product numbers of our suppliers, which none of us actually knows.

So I have begun to add this suppliers' product-number to my products' descriptions and designed a separate field in my database for that text-string. At least in principle my customers might perform a search on that figure. Perhaps I will elaborate that and leave a special note to it below the input field.


 6:52 pm on Aug 17, 2006 (gmt 0)

You may even want to have seperate search fields, or radio controls to toggle the search.

One for "Product Search" then one that is "search by catalog number"

Breaking them up can keep your SQL trim, the catalog search can be a seperate sql statement that only checks that one field.

[edited by: Demaestro at 6:56 pm (utc) on Aug. 17, 2006]


 7:05 pm on Aug 17, 2006 (gmt 0)

On one site (dedicated search page) a client uses three separate search fields, and they are separated into discrete boxes. So, for example, on a book site, one field might be "Search by Author" another "Search by Title" and a third "Search Tables of Contents".

The above approach has worked supremely well. On another client site, they use radio button toggles to switch search types. That client gets more user complaint about site search, and significantly fewer conversions through site search as well.

I know that good programmers like elegance and efficiency - and toggling is certainly more elegant and efficient than dedicated boxes. But I think the separate search boxes approach, clearly labeled of course, is a lot friendlier for the average, non-technical user.

It's been interesting to watch Amazon's site search evolve over the years -- because I know they test the dickens out of their interface. They currently use a dropdown box to toggle search types. But even so, I recently searched over Groceries for a DVD.

[edited by: tedster at 8:20 pm (utc) on Aug. 17, 2006]


 8:10 pm on Aug 17, 2006 (gmt 0)

I agree with that Ted, Many of my sites end up with an advanced search page that is just an Uber form that has all fields covered.

As long as the layout is clean and everything is properly labeled and some easy to read explainatory text then you should be good.


 8:23 pm on Aug 24, 2006 (gmt 0)

Find a provider that allows you to use keywords. Then populate the keywords field with misspellings and related words (so...yoga pants, keywords yoga clothing, yoga apparel, yoga knickers, whatever, etc.)


 9:05 am on Aug 26, 2006 (gmt 0)

jsinger's original question on this thread was:

Do you have any tips to share in helping visitors find offerings?

Somehow it has evolved to "how to improve site search results".

Yes, soundex, keyword loading, thesaurus type matching and all the rest will help, but how about something completely different?

There a neat little toy out. You think of something, then it asks you 20 questions. By the time you've answered the questions, it knows what you were thinking.

Twenty questions is too many ask a customer, but since any given cart only deals with a set number and type of items for sale, I'm sure it could be done with less:

An Aritficial Intelligence Search Assistant, (let's name her "ASIA"), could be a neat "search" feature.

Each question would be displayed on screen, one at a time, subsequent questions would be based on previous answers;

AISA: Hello, are you looking for a gift for a-

[_] Man
[_] Woman
[_] Child
[_] Group
[_] Pet

ASIA: How soon do you need the gift delivered?

[_] Overnight
[_] Next Few Days
[_] 7-10 Days
[_] 2 Weeks or More

AISA: What is budget for this gift?

[_] Under $25
[_] $25-$50
[_] $50-$100
[_] $100+
[_] any price for right item

ASIA: We sell clothing, toys, food and artwork. Do you want to search in one category only?

[_] Clothing
[_] Toys
[_] Food
[_] Artwork
[_] (all categories)

...anyway, you get the idea.

After a few questions, "ASIA" could suggest appropriate items. It is more a process of elimination, ie- items that can't be shipped "next day" may have to be eliminated. Items over shoppers budget, etc..

I would think that done nicely, it would be like having a personal shopping assistant. An animated avatar would make it fun.

This 51 message thread spans 2 pages: < < 51 ( 1 [2]
Global Options:
 top home search open messages active posts  

Home / Forums Index / WebmasterWorld / Ecommerce
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved