SLURP Bots?

Inktomi's Spiders... What are they?

[Eric_Lander]

So after viewing my web trends reporting for a particular client's site, I've found a revisiting spider by the name SLURP. Now, I know it's an Inktomi based spider, but if they no longer spider the web and index it (for searchable keywords anyways…) than what's the point?

And there are two of these Slurp Bots… One of which is Slurp/cat and the other is Slurp/si. I'm not so sure why there's any, nevermind two of them viewing the site.

Are there ways to control what these bots are after (ie: robots.txt, etc.) so they actually will seek out and index anything? In the spiders report, I'm given a URL of [inktomi.com...] to look at…. But see nothing special there either.

Anyone know about these slurping spiders?

[stcrim]

Eric,

Welcome to the INK forum of WMW -

>>>than what's the point?

That's a great question I wish INKTOMI was address here - but don't count on it. In the good old days those were follow up spiders.

I think INK still spiders the web with them just in case the INK-Pay-Mess completely flops. They don't want to have to start from scratch.

I wouldn't spend any energy trying to control them (ie: robots.txt) as they don't do any harm or for that matter any good.

-s-

[Eric_Lander]

Are their presence in the logs evidence of my paid service subscription to be listed within inktomi? If so, what are the two types of bots and why are there two?

[Xoc]

See here: [webmasterworld.com...]