|Wisenut presentation at Boston SES|
big talk from the new guy
Wisenut was represented at the "Meet the Crawler" session in Boston today by Steve Gemignani, their Vice President of Search. (Picture of the panel and complete roster in Brett's Google post: [webmasterworld.com...] ).
Steve joked about being the new kid on the block but went on to build expectations for Wisenut's relaunched site. He said that since acquiring the engine, Looksmart has been rebuilding its infrastructure. Today, they are rolling out their enhanced site and updated index. The updated index contains 1.1 billion documents. Steve asserted that by Q2 they would have a 30 day refresh cycle in place.
In response to various questions to the whole panel about crawler features, Steve's answers indicate that Looksmart is putting significant effort into building a feature rich, highly competitive engine. [anyone who was there want to add specifics?]
Gemignani also claimed that one of their key differentiators is the Looksmart human-powered directory. They use this directory to seed their crawls. Besides this, he seemed to be suggesting that the directory plays a role in their algorithm [I'm mostly infering this. If anyone got a specific quote here, please add it.]
When an audience member asked, basically, why should we care about Wisenut, Steve declined to offer specifics on distribution deals but very directly said something along the lines of "watch for big things from us" (loose quote).
This post is meant to summarize Looksmart's presentation and give those who weren't there the gist of the news. Steve was a bit less reserved in his comments than some of the other crawler reps, but all of the reps tend towards the cryptic. So, check out Wisenut again, judge for yourself and--I guess--watch for big things...
[edited by: jeremy_goodrich at 5:22 pm (utc) on Mar. 7, 2003]
[edit reason] fixed link [/edit]
Nice post camster,
I personally would like to see Wisenut do well, we need a bit more competition in this industry.
The 30 day updates will be a start, and using the looksmart directory (especially the zeal non commercial cats) will add a human touch to the SERPS (depending how it effects the algo of course).
just been checking the index at wisenut, and some results are from sep 2002!
they really do need to sort out their freshness if they want to compete at all
That number gives me pause...1.1 billion? When the came out of beta, they launched with something like 1.6 billion, and at the same time, Google's index jumped beyond that, where previously, it was around the same number.
Google is now well past 3 billion, and when WiseNut launched, numbers wise, it was competitive.
To top that off, Google refreshes their index (3 times that size) every 30 days...and LookSmart says 'they will try to do 1/3 what Google does'...hm, that doesn't sound like good news.
Still, it's not a bad engine...if they really put some work into it. The bit about the servers is very important. :)
Way too slow to catch on to the general surfing public.
The wisenut rep was claiming that they now had a much more robust and speedy platform. I believe he said that the new index was currently 1.1 billion. He did not say this was as big as it would be getting. He specifically stated that they don't handle dynamic pages very well right now and that they are working on this. That would presumably boost the index size significantly.
ZyBorg has been hungrily chowing on my site a lot lately. It does look like they are working on increasing their size.
Hm, so it's now (or going to) be at 1.1 billion.
Speed - that is a good thing.
However, trimming the index down from the 1.6 billion they started at (when out of beta) seems very odd.
But I guess that's what spin is for - to make it sound good, when it's bad. :)
<-- to make it sound good, when it's bad. :)
lol sounds about right :o)
the way I look at it is they *have* to do something if they all want jobs so get your finger out wisenut and get some healthy competition for google.
my 02p worth.
Either spin... or he knows something.
Ya gotta think LS is trying to win an expanded or even exclusive deal at MSN, if not an outright acquisition by MS. Do they have a good enough relationship there that they've convinced MS to give them time to get Wisenut ready for primetime before it makes the move that so many people seem to be expecting.
Pure speculation on my part...
I believe that 1.1 billion was his count for html files - not pdfs, images, multimedia, etc.
Size and Speed equate to ZILCH, without a distribution deal that means something....
They can index 10 billion documents with every day refresh, and use CRAY servers, and it won't mean a thing to most of us, because Wisenut still won't be on the radar in our server logs!
LookSmart has survived because of the MSN deal, period!
They got a lot of negative PR with Webbies, when they pulled their revenue model out from underneath a lot of companies that bought in to the LS directory listing annual subscription.
LS says they have made up the lost ground financially after switching to a CPC model, but after MSN, I hardly consider BlowSearch a quality distribution partner.
|using the looksmart directory (especially the zeal non commercial cats) will add a human touch to the SERPS... |
In the case of Looksmart, human=$, and, if the algo of a search engine that owns Looksmart gives a boost to Looksmart subscribers, there's a skewing of the playing field that might make the FTC take notice. Where then do the paid listings leave off?
We already have the situation where people are buying PageRank on Yahoo, and we not only live with the situation, but some of us use it. If Google owned Yahoo, though, I think the situation would get a little more clouded.
I worked indirectly with Steve years ago when he was at Primedia. He's a good guy and a smart one. I'd like to see him succeed at making Wisenut/Looksmart a big time site. We need some real competition in the SEs. That's the only way the SEs are going to be kept honest.
But Steve, if you see this post, tell your bot to ease up a bit. It is bad webmaster relations to send it out at these speeds. I saw this little guy:
Mozilla/4.0 compatible ZyBorg/1.0 (firstname.lastname@example.org; [WISEnutbot.com)...]
pull down something like 6,000 pages an hour for a couple hours the other night. Today she/he/it pulled down something like 2-4,000 for 15 straight hours. By comparison, Miss Googlebot (I'm convinced it's a she), be she Miss Deep or her twin sister Freshy pulls down no more than 1 every 3 or 4 seconds - 1,000 or so per hour.
If Steve is looking at freshness, a crawl like that every month, while not fatal, is gonna cause some problems. I can't even imagine what will happen if Mrs. Deep and Zyborg show up at the same time every month. Most of the content is dynamic and involves a VB script with 8 queries through MS Index Server, so that many pages is a heavy load.
Great posts here... One clarifying point about the web index size. The numbers quoted were 1.1 Billion fully indexed documents and a web graph of approximately 2.7 Billion. There is always confusion about index statistics, many engines quote index size as documents crawled. The old 1.6 Billion document number related to the size of the web graph.
Welcome to WebmasterWorld!
The clarification is appreciated -> but, as we've been saying, big talk is just big talk, until something happends.
Previously, if you searched, most of WiseNut's results seemed to be derived from spidering other SE's...perhaps the larger Web Graph is a result of WiseNut switching from a meta engine derived base to the Zeal / LookSmart directory as a seed for their spidering.
Time will tell if something happens. Though, after trying more than a year ago to sell family and friends on WiseNut, I'm not going to do that one again till I know that they have at least 10,000 servers and a few more gerbels to run them. :)
I think it is agreed that this horse needs more horse-power. I'm not convinced that there is real room in this category for another so-so search engine. Google and Fast do a pretty good job right now. Anybody who wants to elbow into the front row needs to have something special. And pure quantity doesn't cut it.
To all those people who want more competition and diversity in search engines, don't forget to spend some time trying to do well on WiseNut. :) I think competition is good for everybody, so I wish WiseNut well.
GoogleGuy, wishing is all well and good, but there is a significant problem with WiseNut:
They are slow as dirt. Really. Have you tried it before?
Honestly, if they can't get the speed up to par, they don't have a chance of making anybody switch.
<--- I like what they have, a lot. But, I can't stand it when I'm doing research and I get requests back to me in a minute +
It's unbearable. Google has set the standard there, searching needs to be fast. It needs to happen at the speed of the internet.
I just did a couple searches and they're about as fast as Alltheweb's but slower than Google.