I've seen mention of Google's "FreshBot." What is it exactly? I've gotten the impression that it's a GoogleBot that does a deeper crawl, but less frequently. Is that true? How can I tell if I've been visited by FreshBot? How often does it come around? I did some SEO 2+ weeks ago, and Google still hasn't picked up my new links.
robotsdobetter
4:17 am on Apr 14, 2004 (gmt 0)
Their are two main Googlebots, one GoogleBot deep crawls the web and the other one is FreshBot, FreshBot gets images and caching for Google.
The deepbot uses IP starting with 216 and the freshbot uses IP starting with 64.
It really depends on how many links you have to your site, the more links the more times they are likely to visit your site.
Netizen
8:46 am on Apr 14, 2004 (gmt 0)
Unless I missed something recently there is no longer any distinction between FreshBot and DeepCrawl - this disappeared over 9 months ago. Take a look at msg 20 in this thread [webmasterworld.com]
