| Welcome to WebmasterWorld Guest from 184.108.40.206 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
|Am I missing something here with Inktomi?|
Lots of crawled pages...hardly any in index
Please share your thoughts on this:
I recently saw Inktomi come in and crawl my site--it got through around 3,000 pages before going away.
A couple weeks later (Apr 10 or so) I get a few result pages back through Hotbot...but by a little bit of testing there were only 13 pages or so included in the index.
Why would Ink crawl so much and not include in the index? Or is there a bigger update of the index to come?
Wow...I guess a lot of people are concerned with Inktomi, since I haven't got a reply yet.
Long live Google!
I dont know too much about the Ink algo, however have you tried searching at:
[search.positiontech.com...] using the following command:
that should show up some interesting results.
Yep, interesting results all right.
13 Pages :)
After Ink crawled over 3,000.
That's just boneheaded. What a waste of my bandwidth. This keeps up and I'm going to ban those buggers.
> This keeps up and I'm going to ban those buggers.
Not sure that is a good idea, they may well include you at some stage. Yes, Google Rules!, but Ink (through MSN) still provides about 10% of my SE referrals.
I have not read anything about how they decide which free pages to include, but they currently have all of my small site in their database.
By "keeps up" I mean 2-3 months :)
I'm going to watch and keep track of how many pages they include in the index. As I write, Ink's in my site indexing *again* and is up to 2,500 pages or so *again*.
If these pages don't make it into their index within the aforementionned time period they're history.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved