Webmaster Tools "crawl rate" vs. reality

[littlecubpanda]

In Webmaster Tools there's a bunch of business about crawl rate and the rate your site is being crawled and access by the Google bots.

It's even got some gizmo where you can turn "down" the crawl rate and seems to put the orders of magnitude in "seconds".

Ok -- that's all fine and good -- but then why is it, if my site is CONSTANTLY being "crawled" by "bots" -- does Google utterly fail to update it in the listings?

My site has some old, putrid, garbage title and description that I changed days ago. If this stupid bot is crawling constantly, why doesn't it change faster?

Something changes on the site, and it's not reflected in the listings for days (sometimes weeks or months) yet the stupid thing is being "crawled" hundreds of times a day?

Again, I'm new to all this, but man, it's frustrating how illogical it all is.

[arieng]

garbage title and description that I changed days ago

Any crawl rate that you assign in WMT is at best a suggestion, and that tool is primarily intended to slow down crawl rates not speed them up. Google will ultimately crawl your site at the rate and frequency it chooses. A few days isn't anything to fret about.

[littlecubpanda]

Ok. I don't expect it to be instantaneous, I'm probably just freaking out over nothing. The other sites all eventually changed, though it took forever (a week).

[Robert Charlton]

littlecubpanda - Simply going through the motions of tweaking titles and getting in the index does not necessarily mean that you will automatically move up in the rankings.

Sounds like you could benefit by some good quality, relevant linking... perhaps suggesting that you also need to promote your content, and that your content has to be worth linking to.

"Freely given editorial links" is what Google is looking for. I would start thinking in timeframes of months, not in terms of days or hours. Crawl rate isn't anything you should be playing with.

[pittbug]

While Googlebot may crawl your site constantly, what you need to pay attention to is the crawl rate for those specific URLs that have changed. You may receive thousands of crawls per day across the entire site, but googlebot has to find that specific URL before it can be updated.

I've also sometimes seen googlebot need a couple or a few requests to a page before it makes changes to the web index.

Hang tight, it'll change. If it's really important, you could try giving it more PageRank by linking to it from your homepage if possible. This can speed up the crawl for that URL.

[littlecubpanda]

Hard to say, considering my site is just a couple of pages. I focus on creating '.com's with just a few pages, since people tend to give such priority to '.com's over blogs, news sites, etc. There isn't much to crawl, in other words, it's pretty straightforward. We'll see. It's been about four days, and my old crap is still in the listings and barely showing life within 100 pages of listings... with the changes I made, it should be in the top ten. It's crazy if it's not put up in the top ten.

[lucy24]

with the changes I made, it should be in the top ten

I don't see how you get from A to B.

[GeraniumV]

You think Google is slow - look at Bing. Bing's crawl rate is not super fast.