Page is a not externally linkable
Jordo_needs_a_drink - 9:54 pm on Feb 2, 2007 (gmt 0)
It's now hitting this site's homepage twice an hour like clock work. Every 30 minutes. A few minutes past the hour and a few minutes past the half hour. It's up to crawling the homepage 31 times since 8 o'clock last night. I've also noticed it's doing the exact same thing to one of my feeds. Hitting it twice an hour also. In the meantime my site (that was ranked 7th yesterday morn and in the 700's this morn) is now not in the serps for the keywords it was ranked on. It's getting worse. The feed it keeps retrieving and the home page do have one thing in common... The feed contains random widgets from this site. And its title and links to the feed indicate so (5 Random Widgets). That's all the feed contains, so the feed changes every time it's viewed or crawled. My homepage contains a section that displays 5 random widgets. So even though the rest of homepage (the majority) is not random, it's still different every time it's crawled because of this section. Just guessing, but maybe Google increased the "page changes to often" dial a little too much?
One thing I have noticed is that Googlebot hit the homepage over 10 times in the last 24 hours. It's still crawling the entire site, but it keeps hitting the homepage a lot. It did the same thing in early December.