LSI & Google

where is the link

[stargeek]

There has been talk over the past few months about google using latent semantic indexing in thier algo and I've come to agree recently.
The part of the LSI process that I'm most interested in is the calculation of word cooccurance. LSI websites don't get into this much and I can't find any papers on it by googler's.
Can anyone point me in the direction of a paper or a recent tech company acquisition that connects google to LSI ideas.

[Marcia]

Here's a few

[javelina.cet.middlebury.edu...]

[lsi.research.telcordia.com...]

[cs.utk.edu...]

[www-psych.nmsu.edu...]

Latent Semantic Analysis (again Telecordia)
[lsi.research.telcordia.com...]

Microsoft also
[research.microsoft.com...]

[Marcia]

Word co-occurrence - can you elaborate a little more? There may be something else out there that isn't called the same thing.

Added:

Here's one

[cs.cornell.edu...]

This looks really good. Here's the HTML in the cache, but it's a PDF

Higher Precision for Two Word Queries

[66.102.7.104...]

Constructing and Examining Personalized Cooccurrence-based Thesauri on Web Pages

[www2003.org...]

That last one looks like hitting paydirt.

[cicru]

Is there any proof that the main search engines are using LSI?

I did some tests but from a brief overview, I do not think so.

[trillianjedi]

I did some tests but from a brief overview, I do not think so.

Google certainly is.

Not sure about the others.

TJ

[cicru]

Do you have any data to proof this? I think that LSI really works when your document base is realativley small and non SEO'd.

When you run a search on billions of docs, keywords will take precedence. Considering the way documents are SEO'd today, LSI will not be of much use.

Searches on library books,articles/news,news groups are some what different though. Since the authors did not alter the keyword frequency and introduce semantically related words on purpose.

If there are 1000 documents SEO'd on the word printer, how can google decide to put a document related to ink or hp in it's SERP's when the other 1000 documents are more important to the person who is doing the search.

Can some one explain this to me?

[trillianjedi]

ink or hp

According to Google, "inkjet" is semantically linked to "Epson" (12,500,000 pages).

[google.com...]

Interesting, I didn't realise that semantics could extend to brandnames. Anyone else seen similar in google?

Also "discount" which doesn't really surprise me, but not a good example of effective sematic indexing.

TJ

[cicru]

~ does the tilde instruct google to do a semantic search?

Searching on phone with a tilde brings nokia in the 1st position. Without the tilde, the results are different.

[trillianjedi]

Yes the tilde shows semantic keywords.

Another one - phone¦nokia

[webmasterworld.com...]

Very very interesting...

TJ

[nuevojefe]

Soda gets pepsi, cola and coke but not the coca in coca-cola.

[nuevojefe]

~mike ~tyson gets boxing. He's sort of a brand right?

I was really hoping to find Don King. lol.

[stargeek]

When I mentioned word cooccurance I meant that I am interested in the statistical threshold that will result in two words being tagged as similar.

In other words, how many times does a pair of words have to occur together on the web for google to deem the pair "related". From merely anctedotal observations it appears that level is relativly low and is dependant on the number of times the pair occurs in relation to the number of times the seperate words occur without eachother.

The result of a concrete knowledge of how google determines weather or not two words are related is the ability to manipulate "related word" recognization, adding a whole new level to google SEO.