helenp - 1:05 am on Jan 24, 2004 (gmt 0)

I just read in an spanish site some very interesting thing.

They said, but donīt know why, that when slurp spider the robots.txt file is always like this:

"Mozilla/5.0 (Slurp/si; slurp@inktomi.com; [inktomi.com...]

and when spider rest of site ítīs like this:

"Mozilla/5.0 (Slurp/cat; slurp@inktomi.com; [inktomi.com...]

The diference are in Slurp/si; and Slurp/cat,

just checked, all my spidered robots file are with slurp/si

That means there are two diferent spiders?
and that slurp/cat maybe not started to work properly yet?