Am I missing something here with Inktomi?

Lots of crawled pages...hardly any in index

[Critter]

Hi all:

Please share your thoughts on this:

I recently saw Inktomi come in and crawl my site--it got through around 3,000 pages before going away.

A couple weeks later (Apr 10 or so) I get a few result pages back through Hotbot...but by a little bit of testing there were only 13 pages or so included in the index.

Why would Ink crawl so much and not include in the index? Or is there a bigger update of the index to come?

Thanks all!

Peter

[Critter]

Wow...I guess a lot of people are concerned with Inktomi, since I haven't got a reply yet.

Long live Google!

Peter

[Shak]

I dont know too much about the Ink algo, however have you tried searching at:

[search.positiontech.com...] using the following command:

domain:yourdomain.com

that should show up some interesting results.

Shak

[Critter]

Yep, interesting results all right.

13 Pages :)

After Ink crawled over 3,000.

That's just boneheaded. What a waste of my bandwidth. This keeps up and I'm going to ban those buggers.

Peter

[Mohamed_E]

> This keeps up and I'm going to ban those buggers.

Not sure that is a good idea, they may well include you at some stage. Yes, Google Rules!, but Ink (through MSN) still provides about 10% of my SE referrals.

I have not read anything about how they decide which free pages to include, but they currently have all of my small site in their database.

[Critter]

By "keeps up" I mean 2-3 months :)

I'm going to watch and keep track of how many pages they include in the index. As I write, Ink's in my site indexing *again* and is up to 2,500 pages or so *again*.

If these pages don't make it into their index within the aforementionned time period they're history.

Peter