Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: goodroi
I was thinking about disallowing HotBot completely, but am wondering what the importance of this search engine is. My referrer report for a typical day shows hundreds of people coming form Google, and maybe half google's amount from MSN. I didn't even see Hotbot as a referrer.
What is the importance of Hotbot in the search engine world?
What HotBot spider is that?
Do you mean Slurp, FAST or Lycos (still seen occassionally)? Or is HotBot spidering itself and if so, with what?
So, I guess I better rephrase my question: How importnat is Inktomi and who do they spider for?
How importnat is Inktomi and who do they spider for?
They are (imo) extremely important. Inktomi results are used by dozens of portals, meta-searches and databases across the internet. They are the back-end choice of MSN, Overture and other big guys. Recently purchased by Yahoo!, Inktomi is almost certain to become an even bigger player in the future.