|Having never read a 'research paper' about the Alltheweb algo|
perhaps the answer is they have none?
| 2:11 am on Mar 29, 2003 (gmt 0)|
Having never once subscribed to any theory about the algorithm that drives Alltheweb, I am convinced, at this point, they never had a cohesive algorithm, unlike say, Google or Teoma, whose algorithmic roots lie in academic papers.
Long have I waited to see somebody post a link to some research or some such...and I've waited without success. Without the fortune of somebody else finding the info.
For all the research, spreadsheets laying around, and what not, I've yet to find one piece of evidence that says otherwise.
Perhaps I'm missing the point?
With all the features being rolled out, it's oh so easy these days to ignore that they don't have an algorithmic, academic base from which they are working.
Or am I wrong? Where's the research, where's the paper, what is the hyperlink for the secret to the beliefs of ordering web pages by user query for the Alltheweb, powered by FAST, search engine?
I've seen blatant spam rise and fall, I've seen incredible pages rise to the top - and there has been no cohesion, no strategy, no long term orientation.
Since the Overture buy out, they have been changing. It now appears a destination site, out fitted with bells and whistles to attract those interested in spam reporting. Or URL info. Great. Doesn't make me want to use it, though.
It's easy to see that, in Teoma for example, there is a method to their madness - it started out as research, and grew from there. It started as an idea out web structure -> and then became an engine.
Google much the same, and Inktomi even now has changed, even though they still stuff in those money links of PFI / PPC / XML pages that are blantant, non labeled ads, the core product has risen above all that to become something truly useful.
In the end, I must say: until Alltheweb / now Oversure gets a clue and reorder the results based upon a rock solid algorithmic base, they will get no further than they are now.
Or, am I missing something?
| 9:24 pm on Mar 29, 2003 (gmt 0)|
Yeah we've just yesterday been discussing that:
You know, there's a huge advantage in not basing your algo on lots of public papers: people are never really sure how exactly to spam you.
As said in the other Fast technology is based on research done at the NTNU, Norway's leading technical university. All scientists/PHDs are recruited from there, starting with CEO and founder John Lervik, best PHD in 1996.
So they certainly have a solid scientific background. I would guess there are papers somewhere in the NTNUs libraries, but I have not been able to dig up anything yet.
| 10:37 pm on Mar 29, 2003 (gmt 0)|
>You know, there's a huge advantage in not basing your algo on lots of public papers: people are never really sure how exactly to spam you.
I agree. Best to keep such proprietary stuff secret.
| 2:20 am on Mar 30, 2003 (gmt 0)|
Um, hm. But -> the advantage of having an algorithm is that there is cohesion and order to the results pages.
As yet, I have seen none.
And spam however you view it, does very, very well. Yesterday's cloaking with Alltheweb -> is today's XML feed based CPC program.
So they even have sanctioned 'spam' however you want to view it.
How does that 'help' me as a user...? :)
Oh, and last I checked, their market share was smaller than Google's....that does NOT lead me to believe that such stuff is 'best kept secret'.
Quite the opposite -> if they did drop a research paper, then they would have more SEO people trying to crack the algo, and those who succeded, because they *knew* how to crack it, would be bragging about their consistant success in optimization.
| 3:28 am on Mar 30, 2003 (gmt 0)|
Running ATW searches Jeremy, I definitely see an algo at work. I've never had trouble optimizing for FAST. I think what you are reacting to is that people are finding ATW too easy to beat. From the way I have their algo figured, it is very vulnerable to spammers. My guess is why things seem so random to you is because serious spammers are doing it for money. Since ATW is such a minor player, the spammers are doing what they do aimed at Google. What spammer wants to nail ATW, while they are buried in the SERPs at Google? The reason why SEOs aren't bragging about having beaten ATW is that isn't on the list of things to do. Obviously, this will be a problem because for ATW to get to be a big player, they'll need a more spam resistant algo.
I've run some sample searches. ATW many times even does better than Google on searches where the topic is of academic interest, and thus has little spam and SEOed pages. ATW doesn't do so bad of a job for non-commercial searches. Which, since that is the vast majority of searches, is significant.
| 4:33 am on Mar 30, 2003 (gmt 0)|
I thought I read a post where you said, "SEO - I feel so dirty" and now you sound like you like it.
What gives? "Never had any trouble optimizing for FAST..." great.
Did I say there was a problem with that? Or that I couldn't beat their algo? Nope.
And why would you bother optimizing, when there are products that will manage, automatically your XML CPC feed with them? If the product also managed your ROI then you would be all set, if you were sold something.
So I don't think many, if *any* SEO's that are into commercial goods would seriously waste their time, unless it was the XML program, to optimize specifically for their engine.
Better SERP's? OK. Sure. Didn't Google grow becuause it's results were 'better'...or am I missing something? If FAST results are better for you, great there seem to be a distinct lack of them around here. :)
Back to the topic of the lack of research paper indicating a lack of cohesion, I still think there is a strong case for that. After all, eliminating the SEO effect to the CPC deal (assuming there was ROI, if not, nobody would do anything, ya?) there should be little to no SEO result because people don't target that engine.
In a sense, then, their web data would be 'better' ya?...I still don't see them with a flock of loyal users, steadily gaining ground on Google. And wouldn't all of us 'geeks' here be into Alltheweb if they were the best?
| 5:07 am on Mar 30, 2003 (gmt 0)|
>I still don't see them with a flock of loyal users, steadily gaining ground on Google. And wouldn't all of us 'geeks' here be into Alltheweb if they were the best?
And are people flocking in great numbers to Teoma which has an algo whose roots lie in academic papers? Dunno what you have been searching for, but I have been seeing Google slip in quality, while other search engines improve. FAST is one of them. In fact, I'd say FAST is #2 behind Google.
| 5:44 am on Mar 30, 2003 (gmt 0)|
Hm, Ink (MSN) is by far #2 in my logs :) But yes, in part, I think Teoma also does well because they have a research paper out there :) and Brett mentioned it, after his interview with their founder...
So - I would argue that academic roots = better SE - but taking it one step further, research = better SE. And Google, by far, seems to have the most (current) research articles floating about, with this or that bit in them...patents help to - Inktomi / Altavista I think have the most 2 of any engine (so far...)
What does Alltheweb have, in terms of 'academic' claims to fame...?
| 6:53 am on Mar 30, 2003 (gmt 0)|
>Hm, Ink (MSN) is by far #2 in my logs :) But yes, in part, I think Teoma also does well because they have a research paper out there :) and Brett mentioned it, after his interview with their founder...
MSN totally dominates my logs behind Google. However, as my sites are in Looksmart, I don't know how relevant that is. Direct Teoma hits on my sites are irrelevant. However, looking at the logs ask.com, while behind MSN, is relevant.
>So - I would argue that academic roots = better SE - but taking it one step further, research = better SE. And Google, by far, seems to have the most (current) research articles floating about, with this or that bit in them...patents help to - Inktomi / Altavista I think have the most 2 of any engine (so far...)
Altavista? Are they still relevant?
>What does Alltheweb have, in terms of 'academic' claims to fame...?
Who cares about "academic" claims to fame? What counts is who in the market can best challenge Google.
| 11:44 am on Mar 30, 2003 (gmt 0)|
Not sure why you think there is no strong academic background. As I said Fast has a huge staff of PHDs recruited from the NTNU.
Apart from that I don't think Fast or any other SE cares what SEOs think about them. They care about their product, and they care about marketshare.
If you say you find ATW serps not satisfactory, okay, that's open to debate. Some people would agree, some would not.
But to think ATW would have a higher marketshare if they had academic papers flying around - hmmm.
About SEO targeting ATW it's obviously exactly like rfgdxm1 says: people optimise /spam for Google 90% of the time, and just see how that works out in other engines. It's rather stray spam as far as ATW is concerned.
Anyway, most likely this is a futile debate insofar as the ATW team and algo will be merged with AV team and algo. The process is going to be overseen by Gary Flake, who has lots of research papers out; add that to all the AV papers and patents, and you have a solid pile of papers to read through :)