Forum Moderators: phranque
For the past several weeks Googlebot was crawling my site every 6 days. I mean a full crawl. Then I put in the 'noarchive' tag. The next day GB hit the homepage and the 'cached' link was gone immediately. Then again, so was GB. I did not get my regular 6 day crawl, and I have had much fewer visits overall.
I did notice that Inktomi has been crawling the site much more often. In fact, it has been almost every day for the past 4 or 5 days. I don't know if this has anything to do with the tag at all, but I was getting maybe 1 hit every few months before.
I don't know if this information will help anyone, but since I had some visable results I just thought I'd share.
