>> It just seems to check a few pages and that is it? What could the problem be?
I've been watching Ink closely since early September and have seen similar dismal spidering activity. At the same time, I had one free inclusion site and one paid inclusion site dumped ("dumped" means the free sites pages cannot be found at all in their index and PFI pages are absolutely buried at the bottom of the barrel).
I also have a couple new sites put on the Web in September and Google, Alta Vista, Fast and others have picked them up but not Inktomi.
It looks to me like they're unsuccessfully fighting spam for their lives (because the PFI site they dumped had been in their index for over a year, and is not spam as far as I and all the other search engines are concerned) so they are having to crank up their spam cleansing so radically as to cut off their noses to spite their face. Still, it's easy to find tons of spam in their index.
It could also be that they can't afford the cost of spidering the entire web anymore.
And maybe they can no longer afford the cost of enough skilled technical manpower to maintain their systems and algorithms properly. Being from the programming world myself, I know when the chips are down one of the first things management wants to do is cut expensive technical resources and stop thoroughly testing code before it goes out the door, usually with disasterous consequences.
My window for analysis is very small and based on a few small websites of less than 50 pages, but I'm trying to contribute my best two cents worth toward understanding the problem.
What's Inktomi "guy's" take on it all? Some real facts from the expert would help dispell any erroneous conclusions based on limited information.