Should you always try to increase spider visits?

what volume should you aim for, are there limits?

[fom2001uk]

I've been reading a lot about Googlebot, Freshbot etc and I wondered what the real benefits of increased spider visits were?

Obviously if you're getting spidered, you're latest content is being indexed, but what other benefits are there?

It seems people are obsessing about increasing the number of spider visits, but what should be a realistic goal. Let's take Googlebot, how often should she visit (for say a 200 page site) and how many pages should she vist during a session. I've seen some sites where the number of Googlebot visits roughly equals the number of pages hit. I'm guessing this is bad becuase it's probably only visiting the homepage, then leaving.

I've seen other sites where Googlebot visits dwarf visits by any other spiders. Is this normal?
If so, how would you try to increase Googlebot visits on sites where this isn't happening?

Would frequent page updates be enough?

Apologies for the number of questions but I'm a bit confused and I'm not even sure I should be bothered by this. Is spider visits really that important a metric?

[lazerzubb]

When it comes to how Googlebot is requesting pages there is no norm, this is since they treat different sites differently :)

And the biggest benefit with the "new" googlebot, is how quickly it can index new pages and new content.

And i wouldn't suggest adding content only so Googlebot will think you change your site regulary, always remeber it's not Googlebot who buy's stuff from your site.

>>Is spider visits really that important a metric?

Not really.

[fom2001uk]

"not really"

that's what I hoped you'd say. I'm interested in search engine referrals from humans, not spiders. Spiders take up bandwidth anyway :-)

Thanks

[fom2001uk]

Just to recap.

Spider visits are influenced by

a) the number of pages you have indexed
b) the frequency of updates
c) the PR of your site

in that order

So generally speaking, larger sites get more spider visits (assuming updates and PR are reasonable)

but spider visits do not amount to human traffic, so increasing spider visits is not an important objective.

That about right?

[mcavic]

Quite a while back, I heard that a site that changes frequently gets visited more often than a stale site. I don't know if that's true or not.

Having lots of spider visits is definitely a good thing, because it means that more of your fresh content will be listed, and more listings means more human visits.

But, I don't think you should worry about increasing your spider visits - just work on the size and quality of your site.

[MadHatter]

Can anyone explain why googlebot doesn't spider my entire site? This site was in the index late March, and it had all 47 pages picked up. April, May and June it appeared to be out of the index. Now in July, it appears in the index with a PR0 and only the index page and one other page showing. To date the bot has not spidered the entire site.
Thanks
MadHatter

[rfgdxm1]

Do you have any inbound links MadHatter? Googlebot regularly harasses my site. I'd have thought daily visits to the home page would be ridiculous, but not Googlebot. Just yesterday within 10 minutes it grabbed by home page twice?! Googlebots are nosy creatures.

[MadHatter]

I have two inbound links showing from sites that are PR4.
Today I only have the index page showing when I check the allinurl: in google. Then I get "repeat the search with the omitted results included" and it shows 3 other pages. The sniplet disturbs me as it shows part of my footer instead of picking up text at the beginning of the page.
MadHatter