Reasons for Low Crawl Rate?

[lionstail]

We have two sites that use the same platform- one of them is getting crawled very regularly, and the other one has barely gotten crawled since October. I have no idea what would cause one to get crawled more, while the other would barely be crawled at all.

Any thoughts? Anything we can do to increase the crawl rate?

Thanks!

[cls_wired]

Incoming links from pages with high PR - is the best way to increase speed of crawling.

[lionstail]

Right...but we have the same incoming links that we had 2 months ago, and suddenly our crawl rate tanked- will new links help that, or is it a sign of some other problem?

[cls_wired]

From my experience one link with PR 7 or higher enough for dramatically increase speed of crawling for a short period.

How many pages indexed by google on both your sites?

[rainborick]

One thing that can cause your crawl rate to slow is a poor server response time. If Google starts to think your server has load issues, it will slow down. You can check the Webmaster Tools console Diagnostic->Crawl Stats display to see if there has been a slowdown in your response times.

I speak from experience here. A couple of years ago I changed a site from static pages to dynamically-generated pages so I could automate it. Between the host's poor mySQL performance and my own sloppy programming, it was a dog of a performer and Google slowed down to crawling a handful of pages per day. Retooling my software brought the response times back down to something reasonable (<500ms) and Google restored its crawl rate to "normal" in a couple of weeks.