Rockintom - 3:16 pm on Sep 19, 2000 (gmt 0)

It's probably easier for all to just re-post it here.
Here ya go:

<Date: September 18, 2000 06:56 PM
Author: Liane (BareboatsBVI@surfbvi.com)
Subject: Inktomi Url Submissions - Don't do it!!!

Hi,

I am new to this site but I sure wish I had found it months ago as I see I am not alone in my ongoing Inktomi problems. In March, I launched a new site and after encountering more troubles than I care to remember, I finally got an answer.

I wrote to one search engine after another in an attempt to find out why my site was picked up ... then dropped, then picked up only to be dropped time and time again. I spent hundreds of hours on the internet trying to find a solution to a problem that nobody seemed aware of ... least of all the people who push the buttons to send the canned responses saying things such as, "perhaps your server was down" or "Often the metatags are the source of problems like this" ... and (my favourite), "site submissions are free and not all URL's are automatically accepted".

At first, I was convinced that our host was at fault as we had recently changed hosts. Then I came to believe that our new webmaster had done something not quite kosher and we were being punished. Then I thought it may have been our local ISP as they had experienced trouble with DNS server updating. Finally, the paranoia set in and I became convinced that our site had been blacklisted by Inktomi!

I had the site checked and rechecked every which way from Sunday. I had it rewritten, redesigned, reworked over and over again trying to find the solution to the problem. I couldn't get any straight forward answers from anyone. I was ready to close the doors and go back to the print advertising industry.

Finally, a very kind person at Canada.com took pity on my plight and forwarded my letter to Inktomi. The very next day I rec'd a phone call from Mr. Tim Gilbert of Inktomi's "back end" technical department. Mr. Gilbert took the time to explain what was taking place at Inktomi and why my site (and many thousands of others) would appear and then disappear on Hot Bot, GoTo, Canada.com, 4 Anything, LookSmart, MSN, About.com and many others. He explained that the "submit URL" feature on the various search engines was the source of all the trouble.

Spammers (mostly p*rn*graphy sites) have learned how to circumvent the system and have overloaded the Inktomi database with p*rn sites. In order to combat this problem, Inktomi came up with some sort of algorithm solution which would make the rankings of these sites less relevant in popularity. However, since they do not have any live editors who sort each and every site in their database, their robot is unable to determine what is or isn’t p*rn. ERGO ... anyone using the “add url” feature on the various search engines is automatically penalized in the rankings system.

Mr. Gilbert and I spoke for about 10 minutes or more and although he was very candid and seemed sincerely concerned ... the end result is that there is nothing you can do about it just now.

If you search any sites using Inktomi, you will find that very few new sites appear and that the relevancy of their search results have been degraded to the point of being "less than useful". (At least that has been my experience.) Excite is a good example of a good search engine gone bad. The searches I have done recently were a joke. The relevancy of results I got were laughable.

Mr. Gilbert further explained that the problem was so daunting that their technical department actually staged a one day walk out to protest the problems they were facing and to bring the matter to the attention of management.

Inktomi is (or was) bantering about the idea of introducing a pay to submit programme ... but they sure are taking their time about it. Considering the extent of Inktomi's importance to the e:commerce marketplace, their lack of immediate response and a reasonable solution to the problem is shocking.

In early August, I decided to try LookSmart's "pay to submit programme" and the results were almost excellent. Our rankings in relevant areas are superb and they did a very professional job of editing the description. It appeared in LookSmart's database within 4 days of sending in the submission. It took MSN about 2 weeks to pick it up and the results there were even better! On the down side, it has still not appeared on Excite (nearly 6 weeks now) and it actually dropped drastically in the rankings on Alta Vista and Raging??? It has yet to

[edited by Brett Tabke : note: original author has given permission for posting]