How many crawler visits is too many?

Trying to figure out why so many crawler visits to site

[boyscout]

How many Google crawler visits is too many? or what is an average number of visits by google crawlers?

I ask this because im seeing one of my sites, in January alone get:

433 Visits crawler4.googlebot.com
331 Visits crawler8.googlebot.com
338 Visits crawler9.googlebot.com

To me, that looks rather excessive for ONE month period, and considering the problems im having with the site and google, i.e. NO PR in 4 months, although major links pointing to it etc, im thinking maybe the crawlers are having a tough time on the site?

Its a rather large site, directory listings, with about 5000 resources, and gets updated about 5 times a week.

TIA.

[ThomasB]

have you checked if all sites are crawlable? How many GoogleBot hits did you get from all of their spiders?

[webdude]

Are these "hits" from googlebot or individual visits. I get a visit once or twice per day that crawl about 20 pages per site. That would total around 60 visits or 1200 hits per month. This does not include the random one-time visits that hit the robots.txt then one page.

[boyscout]

How do I check if the site is crawlable? Any spider emulators around?

These are visits by GoogleBots:

Hits Files Visits Crawler
-------------------------
1073 1072 433 crawler4.googlebot.com
602 602 331 crawler8.googlebot.com
553 553 338 crawler9.googlebot.com

Do these look about right?

[ThomasB]

I'd say it's ok, but probably below average. I'd focus on getting good backlinks instead of watching GB spidering.

[jpalmer]

Gidday from downunder all,

Boyscout, I got hammered in Nov/Dec 03, not quite as impressively as you, but then my resources aren't quite as impressive either.

It was enough for me to email G support and I two'd and fro'd with with them over a couple emails, but am none the wiser.

Guess they're just double checking it's not dupe or stolen or hiding anything nefarious that'll endanger homeland security ;->

Just take it on the bandwidth blowout and be thankful you're not in the dance threads whinging about the latest dance and how it's all the wrong beat ... yada yada yada ...

Cheers and hooroo
JP

( Hey! I finally got to use a Yada ... in context! )

[boyscout]

Thanks guys (sorry late response, have been away)...

Still cant figure out why in the world Google just wont PR this specific site, although googlebots all over it.

[Schabroenkel]

Hmmm...

got 300,000 hits of Googlebot in january, costing me about 8 GB of bandwith - _that's_ too much...

Martin.

[jimbeetle]

gets updated about 5 times a week

That's one reason. Spiders love fresh food and keep going back to where they find it just to check if there's any more.

Keep it up.