Will my site ever get indexed in Inktomi

[mbennie]

I have a site that is 2 years old. Thousands of incoming links, about 1000 pages of unique content. Google, Alta Vista, All the Web crawl it religiously.

No barriers to indexing that I can see. Clean robots.txt, .htm extensions on every page. Logical link structure.

Inktomi came by about 18 months ago and crawled the entire site. Did well with MSN for about a week then...poof. Gone.

Since then Inktomi hasn't come by except to grab the home page once in a while.

This is a content site so I really can't afford to pay the 15K for inclusion and I'm getting nervous that visitors won't be able to find it when Yahoo! completes the changeover to Ink.

I have tried using the free MSN/Inktomi submit and get no results. Any helpful suggestions out there?

[Bobby_Davro]

If you do go for the inclusion, just submit a single page to test the water. Your site may be banned in Inktomi and they won't give you your money back either.

I know, cos I just forked out $2500 only to find that the site in question was blocked by Inktomi because it shows a competitor's results (that was the reason given). And then they kept my money as well. CC charge back in progress.

[mcavic]

blocked by Inktomi because it shows a competitor's results

Can you explain further?

[Tim]

mbennie,
Please stickymail me your site and I will investigate
Thanks,
Tim

[Bobby_Davro]

As I understand it, Inktomi didn't like the fact that the site uses Infospace results on it and counts them as competition, so they have blocked the site from being spidered, and that includes even through PFI.

That would have been fine - I can accept that they don't want to promote the competition in their engine - except for 2 things:

1. They kept my money anyway.
2. Infospace uses Overture ads, so it is hardly the competition.

[salmo]

Scary, presumably then sites carrying Ad-sense advertising are also likely to get zapped?

[Bobby_Davro]

I won't pretend to know what is going on in their heads, or what they qualify as competition. I would have thought that they will limit their attention to the bigger names, rather than penalizing smaller website owners, but who knows?

[Tim]

Bobby,
I can ensure you we never ban sites for using someone else's search results. That would be silly and would only hurt our comprehensivenes and search quality in the end.
In fact, Infospace uses results from Overture, Inktomi, FAST and Altavista so they are not considered competition.
If there is a penalty in effect which I cannot confirm without knowing the URL it must be for some other reason such as overoptimization. We place penalties on sites whose sites are using techniques that are designed to make the results show up at inappropriate times and in inappropriate placements within the search results and thereby hurting the user experience.
Tim

[Bobby_Davro]

Here is a quote from the official rejection letter:

<snip>

I don't think that I am misreading that.

If the site was regarded as over optimised I could accept that; as long as I got a refund. I have to say that I am most unimpressed. Even when I told them I was going to do a chargeback they didn't offer a refund, which seems insane. That is just provoking me for the sake of it.

What's more, only a handful of the URLs were ever looked at. They weren't spidered or even looked at by a real person, so Inktomi actually have no idea what is on the pages.

Further to that, there is no such statement in Inktomi's spam policy; the redistributor even admitted that it is in the "unpublished" spam policies.

If you feel that you could help with this and arrange for a refund I would be a lot happier than having to do a chargeback which is bad for both of us.

[edited by: Brett_Tabke at 11:32 pm (utc) on Jan. 22, 2004]
[edit reason] Sorry - no email quotes on WebmasterWorld [/edit]

[mayor]

Bobby, do you mean you loaded your pages with copies of the SERPS from another search engine, in this case InfoSpace?

[Bobby_Davro]

The site in question actually uses several sources of data, including Infospace, Mirago and the Limey Search UK Directory. It is a proper search portal in its own right. We were actually just trying to get some kind of presence in Inktomi because people do actually search for the site name. Inktomi had wiped off every last page of the site, so I hoped to at least provide some pages in the index for people to find.

It is not even as if it was a lot of pages that we were submitting.

[mayor]

>> Inktomi had wiped off every last page

Oh, you mean Inktomi had already booted your site, then you tried to get some pages back in using PFI?

You would think they would have rejected your purchase in that case, huh?

[Bobby_Davro]

I didn't know whether the site had been blocked or not; there used to be some pages in there, but they all vanished. You would think that they would simply have rejected the application straight off, yes, since they had no intention of including any of the pages.

It is a concern for all sites that don't currently feature in Inktomi; how do you know whether the site is banned, or whether Inktomi hasn't indexed the site?

In my research on this, I have noticed a lot of sites limited to just 2 pages in Inktomi; I would imagine that this is a deliberate cap on the number of pages spidered. I would even have been happy with just that for our site so that searchers could find the front page.

[7Sins]

Bobby_Davro

Your story, attitude and treatment pretty much mimic mine. I asked politely for my problems to be looked into and received only canned responses.

You ask for assistance and get given canned responses and are treated with utter contempt till you get angry and do something about it.

One of my pet peeves is when I pay for inclusion I expect inclusion! If Inktomi judges me to have the worst pages for those keywords then make me the last result of 2million. At least include me in the index.

The final straw is that even though those pages are not in the index, the reminder notices to renew them keep coming strong. Do you really think I am going to pay the renewal and throw more cash down the drain?

[ByronM]

If this is indeed true i hope Yahoo holds off on switching to Inktomi..

Inktomi results are HORRIBLE for everything i look for :(

[7Sins]

ByronM
I have seen some excellent serps on Inktomi, however I don't believe great serps are as profitable for them as what you normally see as those indexes usually only last 4 or 5 days.

[panic]

Since then Inktomi hasn't come by except to grab the home page once in a while.

Was your site ever indexed in Inktomi? Were there tons of indexed pages in Inktomi that got dropped?

If the site was regarded as over optimised I could accept that; as long as I got a refund. I have to say that I am most unimpressed. Even when I told them I was going to do a chargeback they didn't offer a refund, which seems insane. That is just provoking me for the sake of it.

I don't blame Inktomi for not refuding the money. The Inktomi TOS were violated, and true to their warning, action was taken.

[kanetrain]

I agree with most everyone here. Ink has some problems. I really hope that they get things figured out before they roll them out to the public over at Yahoo.

[Bobby_Davro]

I can ensure you we never ban sites for using someone else's search results

Tim, you never got back to me on this issue. As I stated, the Inktomi partner company sent me that letter detailing the fact that Inktomi had specifically banned the site because it showed a competitor's results. I understand that you think that this shouldn't be the case, so can you get the ban lifted please?

I presume that there won't be a problem doing that, since Inktomi has a policy of not doing what the partner company says they are doing. The site therefore shouldn't be banned. I will of course stop the chargeback on the credit card if you would be kind enough to sort this issue out.

Thanks.

[ravishingr]

i have a series of sites, and i paid to include about 15 pages of one of my sites in inktomi about a year ago. Now, one year later only those 15 pages are indexed in inktomi from that site, but all of my other sites that i never paid to submit have been fully crawled and indexed. What's up with that?