Googlebot visits EVERYDAY - why?

[rtsit]

A new site that I've created has googlebot visiting every single day. I don't have high rankings (infact, last time I checked, I wasn't even in the index).

Why does googlebot return everyday? What is it looking for, what is it hoping to find?

Am I considered a SPAM site, and googlebot keeps returning in order to see who I link to (to lower the ranking of any sites I link to).

Or, am I going to be ranked extremely high one day, in the next few weeks, and googlebot can't believe how good of a site I am, and as a result, it must re-visit me EVERY SINGLE day?

Again, googlebot revisits my site almost every single day, and I can't figure out why... Even if I change the content, I still get daily googlebot visits.

HELP!

[deejay]

Send her my way!

Actually I don't know why she'd visit daily, unless you are adding new content frequently? Ms G does like to pay more regular visits to sites that update frequently.

I very much doubt it's the spam thing.. SE resources tend to be spread thinly enough that if you're identified as a problem site they are probably LESS likely to visit you, not more.

On the upside, if you're not already listed in Google, then it's an indication that you will probably be in the next update.

[soapystar]

yes..for 6 weeks google cme to see me every day!....then one day i deleted my files and republished them and the next day and since then i have been deleted from the google cache and serps!..i wonder if its checking that your files are staying the same....and if you replace everything with a new creation date it thinks you are playing games....then chucks you out?...or can i hope to make a return?..like mike tyson?

[diddlydazz]

As to why Google visits some sites more frequently than others has been a hot topic in the past.

I personally think it is to do with the 304 codes that are returned.

But the main reason behind it IMO is just for the Fresh tag (as it used to be).

All of my sites (AFAIK) have googlebot visit daily or once every couple of days and I have never had problems.

Nothing to worry about IMHO

Dazz

[jdMorgan]

soapystar,
After discovering a problem common to all of my pages on one site, I replaced almost every html
page on that site. I did not delete them, just overwrote them. Googlebot didn't seem to mind.
It's possible that you got dropped because Googlebot visited before you put up your new pages and
while pages were missing. Barring any other problems with your site content, you'll be back in the
index soon, IMHO.

rtsit,
I've noticed that our new pages tend to get a lot of attention from the 'bot. Rumor has it that
Google is working on methods to identify new and updated pages quickly (See employment info on
Google's site). So Googlebot may be trying to figure out the update frequency of the pages on your
site. If your site is clean, then consider yourself lucky for all the visits!

Jim

[soapystar]

jdmorgan....hi!..i cant help thinking my mistake was deleting then republishing the new pages instead of overwriting!....googlebot had visted since so im hoping im back in next index!

[luma]

Hi rtsit, welcome to WMW. Are some of your pages listed in DMOZ? I am pretty sure Googlebot visits these pages more often than others (though not necessarily daily). I wouldn't worry a bit (unless you play foul). :)

[luma]

I just checked some stats from a PR6 site (it's approx. 60 pp) and it seems Googlebot really likes it:

July 1: 5 pp; July 2: 6 pp; July 3: 40 pp (full crawl); July 4: 2 pp
July 5: 10 pp; July 6: 17 pp; July 7: 0 pp; July 8: 2 pp
July 9: 16 pp; July 10: 3 pp; July 11: 17 pp; July 12: 12 pp
July 13: 28 pp; July 14: 16 pp; July 15: 18 pp; July 16: 35
July 17: 63 pp; July 18: July 19: 30 pp; July 19: 26 pp; July 20: 36 pp

This always includes robots.txt

This site is rather static: 15 pp were updated on July 1st, and four pp were updated later this month. All other pp are even older.

This really looks as if Google implemented a bi-weekly full crawl cycle. Can anyone confirm?

Go Google Go!

[jdMorgan]

luma,

I've had Googlebot stop in once every three days (on average) this month and grab robots.txt
and "/". This is a small PR5 site. Last full crawl was 19-July. I suspect that new sites,
frequently-updated sites, or higher-PR sites may get a full crawl more often.

soapystar,
Yes, Murphy's law states that the probability of a Googlebot visit increases if your server
is down or if you have temporarily deleted any files... ;)

Jim

[guezo2]

My site's homepage went from PR4 to PR5 by the end of June.
In June, Googlebot used to visit my site everyday but during this month, Googlebot didn't come :-(
My server hasn't have any failure.
Does someone else noticed the same problem?
Maybe Googlebot will come back this week during the Update?