Welcome to WebmasterWorld Guest from 184.108.40.206
Forum Moderators: bakedjake
Right now the Excite portal is still apparently operating, but they no longer crawl sites, have submission, or their own search results. On the SERPS the "Submit your URL" link goes to Overture for their PTP listings.
The search results appear to be identical to what's seen at Overture (without prices), which uses Inktomi to supplement their own paid listings. There's no differentiation, so that's the assumption. For certain, Ink results are there. Doing a search using Excite's Directory option doesn't give individual listings, but does give a list of ODP categories related to the search phrase.
As recently reported and discussed here, Hotbot has also quietly eliminated submissions, which used to be for Inktomi inclusion.
Sites that are still including those in their submissions lists, as well as Northern Light, which also no longer takes submissions, are out of date and either unaware or just haven't gotten around to updating their information yet.
I assume we should not look at these engines as players anymore. Currently, it seems like Google and Looksmart are the order of the day, with Inktomi also being a player (these are the ones I keep hearing about in here)
Marcia, from what you say, it seems like pay for spidering is the road that SE's are going down. Sorta sad really, and has a close tie to Tedsters thread (http://www.webmasterworld.com/forum9/2384.htm) about sites moving towards a subscription based medium to cope with this change in web landscape
Ink has been teetering on the edge for at least 22 months (remember the big Ink crash of 2000). Their databsae was full of porn spam, good sites were dropped and their algo was in serious question. Submitting through Canada.com, which at one time was a sure fire way to be included in Ink, became futile.
There were hints back in June or July of 2000 that they were considering pay for inclusion and had targeted September 2000 for the new plan, but that didn't come about until late fall or early winter of 2000/2001. They have never fully recovered from that and I think it was the ultimate undoing and the cause of the slow and painful demise of HotBot and many others. People began searching for better results, opening the door to newcomers like Google.
Everyone raced around looking for other (uncorrupted) databases to power their results. Ink eventually lost Yahoo, which went to Google in what must have been a huge blow to Ink and was certainly a huge boost for Google. GoTo, came up with the PPC concept, irrevocably changing the landscape of the search engine world. MSN and AOL stuck with Ink ... but began making deals on the side with LookSmart and then GoTo, just in case Ink fell apart completely.
Companies like Excite, Alta Vista, Netscape, Direct Hit, Ask Jeeves, HotBot, etc. wallowed around in the search engine doldrums for a good year and a half with stale databases, irrelevant SERPS and ineffective management, while Google gained more and more ground with fresh indexes every month and more features to serve the general public and webmasters alike.
It comes as no surprise that Google is where it is today. They ran a lean, mean, data gathering machine with an excellent algo while all the others looked around in desperation, trying to sort out what to do and chasing their tails while losing business every day. LookSmart has been quietly regaining its former strength, Yahoo is trying many (IMHO) ill conceived ideas to boost traffic while client services quickly wane.
MSN and AOL continue to battle one another while (seemingly) ignoring what is going on around them but somehow manage to keep an iron grip on their customer base ... loosing some here and there to Google ... mainly.
Now along come Teoma and Wisenut, with great hopes for the future and poised to give Google a run for the money. Nope ... both sell outs and also rans. Perhaps Teoma has a good chance of becoming important. I do not hold out the same confidence in Wisenut as others here do. I feel they are a flash in the pan ... a bright star that will come hurtling down and burn out before it goes anywhere at all. (Just my opinion)
Fast and Lycos are like the distant cousins who keep trying to break into the North American market ... but for some reason, they just keep missing the motherload. Nobody really understands why. Good algo, good results. Good customer service. There is still a lot of hope for these two ... but my guess is that few here would invest large sums of money with them. They seem to make promises they can't keep and time is the one thing that searchers don't have much of. People are inherently impatient and its taking them too long to get their act together.
When I think back two years ... I believe it was the spam attacks on Ink and their inability to deal with it and clean up their database in a timely fashion which began the whole domino effect of all the dot.coms.
It points out the importance of not having all your eggs in one basket, which makes me worry tremendously about the dependence on Google.
The horse race is far from over and I imagine there will be even more major changes to the landscape and some very big surprises before year's end.
Hey Matt ... when did you say Gigablast would be out of Beta???
GOOD QUESTION!!! This guy is a potential SE messiah.... What would happen if more talented scripters got off their duff and started to give it a go?
Whatever happens with GigaBlast it has already proven that the battle for relevant FREE SEARCH ENGINE submission and results is far from over!!!!
Power To The People,
Realisation of the power of the Internet has yet to happen. However, once it does, then the Web can finally continue to grow, without major investment. I'm sure Lycos won't close Hotbot or turn it into something it should not be. Everyone is waiting for an up-turn in the market. Goverture took advantage of, and maybe even to an extent caused, the slump in the market.
I think we should all prepare for a lot of work to come!!!
On the web we have dozens of search engine databases all looking for the same paid laistings. We used to call them "Search engines" but I'm thinking of them more and more lately as the old "commercial directories". AOL. MSN, much of INK.. that it the way it is moving - soon they will look close to the real overture.
There simply is not enough room for all the competitors in the field. Web site marketers are just not willing to fork out money year after year for an unknown return to dozens of different databases. However they may be willing to pay a substantial amount to directories specialising in their field.
What in syaing i guess is a listing in the yellow pages is pretty cheap, but in an industrial directory is much more. Many search engines are not almost like yellow pages pretending they are industrial directories.
Im predicting a shake out - maybe a burn out too as you say. Some will be wrying the day they left the real search engine market to Google, and they will have to buy their way back in, but this time Google will be holding all the cards and the ante will be way up.
just checked out hotbot and lycos, looks like they have totally given up the ghost on maintaining their own DB; on my standard "test" searches in my industry looks like the results come from ODP.
Pity, I really liked hotbot's results options and as an alternative to google to pick up "deep link" and lateral subject pages/sites.
So with everyone plugging into google and/or ODP what are we going to do when they go PFI or PPC to recoup the VC and give themselves the lifestyle they aspire too?
SEs are like buses aren't they? There's sure to be another one along in 10 minutes!
P.S. I might be more inclined to PFI, if I could [among other factors that might make me reach for my plastic ;) ] determine the frequency of the spider visits myself. But every 48 hours for 6 months or a year? PULEASE! Waddya think I am, CNN?
did you know that you can still submit to inktomi from hotbot uk? [hotbot.lycos.co.uk...]
no idea if it actually works as i've only just found it - i didn't bother looking at hotbot when i read here that hotbot had withdrawn the site submit ...