Welcome to WebmasterWorld Guest from 54.166.114.43

Forum Moderators: open

Message Too Old, No Replies

Spider New? or not

Never seent this one before

   
1:15 pm on Aug 30, 2000 (gmt 0)

10+ Year Member



hey,

Ran in to a new (for me) slurp.... Agent name Slurp/cat

Anyone seen this before??

6:40 am on Aug 31, 2000 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Too the best of my knowledge Inktomi was the first search engine to perfect directory building from spidering. Huh? They run a spider and then build an ODP/Yahoo style directory out of the data. They can do it on the net wide, or build specific niche directories at a customers request. It is very intelligent software and was some of the basis for my Themes article because it *is* a theme'ing spider/engine.

The /cat spider is the Catalog or directory spider. It is a good thing to see on your site. You have been choosen for one reason or another to the /cat treatment. This could mean big hits down the road. The only problem at this point? very few companies are using the Inktomi generate directories. They are very small at this point - I don't know of a single public one not behind a serious firewall (they have built a few for corporate intranets) and the technology is a year old this month.

9:54 am on Aug 31, 2000 (gmt 0)

WebmasterWorld Senior Member nffc is a WebmasterWorld Top Contributor of All Time 10+ Year Member



LookSmart and Inktomi Form Alliance to Power the Vertical Portal Market [inktomi.com]

Old news but could be relevant.
[marker brett]

12:32 pm on Aug 31, 2000 (gmt 0)

10+ Year Member



Brett. NFFC

Thanks for the input. I wonder if this Directory will mind cloaking :) All pages followed are cloaked so we shall have to see. I am happy to be hit by that spider.

Thanks again.

Dr. Bill

1:15 am on Sep 2, 2000 (gmt 0)

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Thanks for the link NFFC. I am still looking for the one from Inktomi from Sept(?) of last year where they announced the directory building software.