homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

Googlebot not very active compared to competition..
Loosing freshness?

 9:59 am on Mar 23, 2005 (gmt 0)

I maintain a large network of websites in alot of different areas so i have quite good insight in crawling activity between different search engines.
I find it strange that googlebot is so passive.
A normal day i get about ten times more bot hits from MSN spider and about five times more bot hits from yahoos spider.
I also see that MSN is much faster including new sites / pages in then google.
Is googlebot crawling "smarter" or are they loosing the freshness of their index..?

Would be nice to hear others experience.



 11:24 am on Mar 23, 2005 (gmt 0)

Google fetches well linked URLs much more often than not so well linked URLs.

I see some getting fetched daily, and others not fetched in over a year.


 12:42 pm on Mar 23, 2005 (gmt 0)

I got some very well linked sites that are getting fetched daily by google but the number of fetched pages compared to especially MSN and also Yahoo are much lower..
so the point is that both msn and yahoo includes more fresh pages faster then google the way i see it..


 4:49 pm on Mar 23, 2005 (gmt 0)

I always thought, frequency of Googlebot visits depends on PageRank and number of quality inbound links on other high ranked pages. Higher the PR and inbound links greater is the frquency of visits. I am damn sure the WebmasterWorld has a lot more visits and pages get much faster, because of its high PR and quality inbound links. PR5+ sites do usually get visited everyday or atleast every alternate days.


 4:52 pm on Mar 23, 2005 (gmt 0)

And not just the frequency of visits but the depth of crawl in your site and the number of pages indexed are also connected to PR.


 5:06 pm on Mar 23, 2005 (gmt 0)


Google seems remarkably fast respidering my site, usually catching textual changes within a few days at most. And I don't see a serious bandwidth issue from any spiders though I try to turn off some of the more marginal but CPU-expensive features on pages when I detect that the visitor is a spider rather than a human.

However, Google seems to be lamentably (>6 months?) behind on spidering images, which from what I hear is not untypical.



Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved