Forum Moderators: open
A new generation of scrappy search engines is emerging to challenge the dominance of mighty Google.
[wired.com...]
A couple of questions about Teoma, Wisenut.
1. Where are they getting their feeds from, as Teoma looks like a redressed Ink ?
2. How often are the crawl cycles ?
3. One keyphrase that comes #1 on nearly every other search engine, Fast, Google, INK, ODP, MSN, Excite, Lycos, AV, the list goes on, and not even on the first page or second - why ?
Read the article, we will see how it materialises over the next 12 months, to see if the aforementioned engines will make the big time.
Let me answer a few of those questions for you :)
1. Where are they getting their feeds from, as Teoma looks like a redressed Ink ?
Teoma has a proprietary database, which has been building under their own name since last summer. They have gotten out the gate quickly, but if you spend some time at their engine, you might notice that the size of their db leaves something to be desired, however, their relevance is superb. I would expect them to be a major player with the right PR moves.
2. How often are the crawl cycles ?
Teoma is probably using a registration database, and following links, as they have no way to submit your url. Their spider cycle, thus, can't be determined by testing, but from what I've seen in log files, it seems to be a lag of about a month. Their bot bears the same name as the engine, Teoma.
For Wisenut, I haven't seen an update since I submitted some pages, but I got a glimpse of their bot, Zyborg, in my logs within the week. If they would refresh their db, or add the pages I tested I would have more info. We had a thread here about that, but nobody reported the time lag between submit, spider, and index...
3. One keyphrase that comes #1 on nearly every other search engine, Fast, Google, INK, ODP, MSN, Excite, Lycos, AV, the list goes on, and not even on the first page or second - why ?
Again, I would attribute this to the size of Teoma's db. They have done a lot in a year, and it seems that they are using technology from Rutgers University [cs.rutgers.edu] and since they are so new on the field, I expect for them to scale their algorithm to the size of the other players like Google, Fast, and Ink, it might take some months yet.
Hope this helps with your questions. It seems funny to me, but Teoma of them all seems to be getting the most attention, even though Wisenut is 3 years old. I don't know why that is, but I think it might have something to do with Teoma having a linux/Apache server set up, as the folks around here seem to be largely on the open source side of things (myself included).
Teoma is only boasting a database of 100 million pages which is pretty small for a spidering engine. (compare this to wisenuts 800 million and googles 1 billion)
It will be interesting to see how their algo evolves. ;)
