Clustering now? FAST and Lycos

[Marcia]

I'm examining results on several search terms for some new pages that were added to a site and submitted to both Lycos and FAST early this month (September). The site is right there for all the search terms, and has been for several days now. At alltheweb it's the index page that's now ranked for the terms, with only 1 or 2 of the new interior pages listed farther down. I'm assuming that's due to the added bit of text on the index page, with links to the corresponding interior pages. There's a good mix of sites listed.

I looked further, since FAST had what seemed to be a clustering problem, for one of my sites that a few months ago had about 7 of the pages listed for the main keyphrase, ranging from # 4 to # 13 or 14, if I remember correctly - essentially hogging the category, which really didn't look right or give a good search result. In double checking today, all the interior pages are not listed at alltheweb, only the index page at # 2, which is good, it makes for a good search. no complaints at all on that.

Not the case with Lycos. where there are 9 of the site pages listed under the main keyphrase - index at #2, the rest at 4,8,9,10,11,14,16, & 25. It's giving a lot of exposure, but isn't really a good representation. It kind of looks "fishy", which it isn't. It's a Lycos problem.

Getting back to the first site mentioned, I checked results at Lycos, where the site comes up for all the search terms, with a mix of index and interior pages. For 2 search terms, it's the index and interior page appearing, for another 2 phrases it's just the index page. For the fifth phrase, not the index page but 2 interior pages are #13 & #14, with most of the preceding listings belonging to one site, having the same exact title and description on all their pages; just a differnt make and model I suppose. Not a problem at all if it's all different sites, but a great number from the same site, essentially for the same thing, is not really giving a relevant search.

I'm assuming from this brief analysis that the clustering issue has been resolved at alltheweb, but that it is not at all the case with Lycos.

Interestingly, on this second site (the older one), in looking at the stats, the traffic from Lycos has moved in right behind Google and google.yahoo. What I'm pondering is that assuming that AltaVista and Lycos are probably the most well-known search engines for a while, that since AV has gotten so incomprehensible, maybe people are using Lycos a bit more. I can't figure why else there would be a noticeable increase in the Lycos traffic otherwise. In fact, while very little, even sympatico.ca did a few more than AV.

This was just a couple of sites I looked at, anyone else notice anything related?