New FAST feature? Clusters?

Just noticed when checking URL reporters

[awcabot]

I just checked the Fast's report on my URL. I checked to see what is said about some of 775 (mostly forum) pages that are in Fast's database and saw at the bottom of the report a series of "Clusters" of keywords.

These look like keywords related to my site, keywords that appear close together. I suppose it would be good for me to start targetting those keywords as well, but I wonder what is the story on these "clusters"

Reminds me a Google lab of "clusters" but it never did much sense to me.

[heini]

That's Fast Topics, a technology they have for years (2 I believe).
It was built using the ODP taxinomy.
There are more engines offering a topicalization as additional search feature.
Lycos.com has shown the Fast Topics on top of the serps, not sure if they actually have them.

[Trodda]

I just noticed these to. Any idea how they come up these clusters Heini? (not quite sure what you mean when you say its based on the Dmoz Taxinomy...)

Thanks
Trodda

[Rumbas]

Trodda, they take the category structure from Dmoz and use that as topics. I guess they include the "cats" where the cats include the search term - or least a synomym for the search term. Pretty neat actually.

[Trodda]

OK, thanks Rumbas. Do you know if they factor the relevancy of the clusters into their ranking algo?

[Fiete]

Thanks for the info, awcabot.

There exist so many differerent methods and algorithms to create *clusters*. Does anybody know which methodology FAST is using?

[Rumbas]

The clusters being built on ODP taxinomy, I'd say they factor that in the ranking = a listing on ODP is good for your rankings.

>which methodology FAST is using

No, only FAST knows. Anybody can take that ODP dump and slize and dice that to fit their needs. Who knows what the guys at FAST do with it?
- I'd like to know tough ;)

[awcabot]

I do not think it is based on DMOZ taxonomy.

This has beeen recently implemented, or rather, I noticed it just recently when Alltheweb crawled most of my site (I did not even know I had a site 775 page big - mostly old forum postings). I think that there is an algorithm that checks what words appear to come in "clusters" or groups from within each site.

Some specific examples from my site

year, camping, heard (3)

There is no way that these could have come a DMOZ taxonomy, but Fast can find these three words in some of my pages, therefore an attempt to find a theme to site. I suppose that the less clusters a site has, the more targeted (or narrow) it's scope.

Webmasterworld clusters are:

yahoo, help & tutorials, promotion (6)
search engine submitting, internet, uk (5)
server side, internet, programming (4)
tips, resources, internet marketing (4)
google, search engines, searching (4)
registrering og positionering, internet, edb (4)
development, macintosh, systems (3)
cloaking, promotion, web design & development (3)
webmaster resources, web design, internet (3)
chats & forums, writers resources, arts (3)
log analysis, internet, software (3)
looksmart, grub, acquired (3)
webmasterworld forums index, search engines (3)

these are almost all synonyms (except to us that is :) ) therefore can give an idea of what webmasterworld is all about, without having to actually visit the site. Note that this is different than a list of keywords from the site, but rather terms that often appear together.

[Rumbas]

awcabot, I'm quite certain that they use ODP.

Take a look at:

registrering og positionering, internet, edb (4)

The exact same words appear in this ODP cat [dmoz.org].

Same goes for:

webmaster resources, web design, internet (3)

ODP category [dmoz.org]

Tells me there's some ODP stuff going on there :)

[IITian]

I am not listed on ODP and my pages were clustered nicely. My guess is that it used my website file directory and keywords in the title to do so.

[Fiete]

Probably FAST uses for the clustering a lot of different available information sources.

I'm quite certain that they use ODP.

If the cluster engine finds in the results several pages that are listed in - and linked from - an ODP category, it is very likely that this information is used for the cluster. That does not necessarily mean that they use the ODP taxanomy. It may get into the calculation indirectly, but it's also likely, that they use it not as the only method.

Btw, I think I have heard that they use also some semantic logic in order to build their clusters.