"Did You Mean" database

Where can I find one ..

[getxb]

Hi,

I have been asked by my Boss to download and try out a reasonably good "Did You Mean" database. It will act as a suggestion tool for site searches .. so if you type Helllo it will suggest the right word which is Hello. My question is is it available for free in the internet? Where can I find one?

What I found is a "COM ActiveX Server Component" which we don't need.

Please help me out.

Regards,
getxb

[stajer]

The database itself is generated by you - it contains a list words that relate to your content. Then you take each search and compare it to that database. If the search contains a word or phrase not in your db, you present them the closest match in "Did you mean..." format.

[Demaestro]

You should check out databases that have a soundex module.

It is basically a "sounds like" function. That coupled with a dictionary lookup should be enough for someone to make a custom app that would do what you want.

I doubt you will find something for free, off the shelf, that does what you want.

[Tapolyai]

May I suggest that instead of downloading a database you calculate the "did you mean" on the fly?

You can use soundex, metaphone, or similar concepts.

What you will need is your language database of words, and the selected phonetic algorithm results.

Since this is for your web site, you can even create this through a program, and periodically update it.

This would allow you to create the "did you mean" results, on the fly.

That is, let's presume you select soundex as the algorithm.

* Go through all pages of your site and create database of words and soundex (filter non-relevant pages, duplicates, stop words, etc.)
* Primary key would be the soundex.

* user enters "helllo".
* Calculate soundex.
* look up matching soundex in DB.
* display words as "did you mean" with appropriate links to pages...

edit: eh, Demaestro already said while I was typing it up. :D

[edited by: Tapolyai at 7:40 pm (utc) on Dec. 6, 2007]

[getxb]

Thanks everyone for your replies. Didn't thought of the "did you mean" on the fly concept; will ask Sir to have a look at this thread so that I can pass this to our senior programmer for development.

Does anyone know of any other possible ways? And is this the way the search engines work too?

Regards,
getxb

[getxb]

Is there a paid version of something like this we can purchase? OR Is there any libraries / modules that we can purchase which will help speed'n up the process and make it more correct?

Kindly suggest.

Regards,
getxb

[volatilegx]

Here is some soundex info... [creativyst.com...]

The above link includes several source code examples in various languages.

[getxb]

Pls have a look at [discuss.joelonsoftware.com...] where someone mentioned that .. "Trying to make your own poor Soundex is asking for trouble." Also Commented .. "It might be nice to make a dictionary (for that proper spell checker library you use) based upon recent searches, or other domain knowledge (such as an index of content on the site). Obviously this dictionary needs maintaining, and old search terms expired periodically etc."

Right now we do maintain a dictionary and we have two heads who monitors the site searches and enters the misspelt words and their suggested words. We now plan to automate the same and enhance the whole procedure.