Keyword Density / Themes

[novatech]

In my infinite uninterestingness, I have been pondering over how an automatic spider / database works out themes without it being directly related to keyword density.

You hear all the time of technology that is able to relate content on the basis of themes / context and not keywords.

How on earth do they do this - at least without having had some poor guy sit down in the first place and at least manually map some correlation between keywords?

Just a thought...

[trillianjedi]

I doubt this has anything to do with density - how would that relate to a theme?

It's more likely in my opinion to be the way that keywords are clustered together in multiple two, three and four keyword "phrases" that determines theme.

TJ

[netnerd]

I would agree. Themes are probably determined by common clusters of words on sites where the shared word in the cluster is the theme.

Blue WIDGET Ticket
Red WIDGET Shop

Where the theme is Widgets.

[novatech]

but is that not simply keyword density?

surely on a page with

Blue WIDGET Ticket
Red WIDGET Shop
green WIDGET opinion
red WIDGET opinion

in the above the main theme would be widget, with perhaps "widget opinion" and "red widget" coming a close second.

The "theme" or "automated taxonomy" as the companies selling it like to call it, looks exactly the same to me as keyword density.

whereas:

Blue WIDGET opinion
Red WIDGET opinion
green WIDGET opinion
red WIDGET opinion

in the above the theme would be "widget opinion".

I can't work out where the complicated bit about "automated taxonomy" is.

[trillianjedi]

I'm not sure that the best example has been sited, but in any event I think maybe you've answered your own question?

You've highlighted this:-

Blue WIDGET Ticket
Red WIDGET Shop
green WIDGET opinion
red WIDGET opinion

Which is a cluster of words. These are three word phrases.

Within that you could use keyword density but I don't really see the point. You may as well use phrase density - it would be more relevant.

TJ

[novatech]

When I referred to keyword density I meant density of keywords, whether as a phrase or single keyword (exactly as if you used a keyword desnity analyser when optimising a website).

Over and above such analysis I can't see where the technology comes in to improve it - it's the whole Java Island versus Java Programming argument.

How would you get it to categories it automatically other then using keyword density?

[trillianjedi]

When I referred to keyword density I meant density of keywords, whether as a phrase or single keyword (exactly as if you used a keyword desnity analyser when optimising a website).

But your example actually doesn't do that does it?

The words "red", "widget" and "Ticket" and "shop" could appear elsewhere in the page without appearing together (non-target phrase).

If you just analysed the page you would get misleading results.

Your example answered your own question. In making the example you grouped the words into phrases.

TJ