Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: goodroi
I was thinking about disallowing HotBot completely, but am wondering what the importance of this search engine is. My referrer report for a typical day shows hundreds of people coming form Google, and maybe half google's amount from MSN. I didn't even see Hotbot as a referrer.
What is the importance of Hotbot in the search engine world?
joined:July 21, 2000
What HotBot spider is that?
Do you mean Slurp, FAST or Lycos (still seen occassionally)? Or is HotBot spidering itself and if so, with what?
So, I guess I better rephrase my question: How importnat is Inktomi and who do they spider for?
joined:Sept 26, 2001
How importnat is Inktomi and who do they spider for?
They are (imo) extremely important. Inktomi results are used by dozens of portals, meta-searches and databases across the internet. They are the back-end choice of MSN, Overture and other big guys. Recently purchased by Yahoo!, Inktomi is almost certain to become an even bigger player in the future.