homepage Welcome to WebmasterWorld Guest from 54.166.53.169
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Why has googlebot stopped crawling site since april 9th
Carwls every day on our other site.
narsticle

5+ Year Member



 
Msg#: 29055 posted 1:59 pm on Apr 15, 2005 (gmt 0)

Hi all and thanks in advance for any help you can offer. We own two sites. One older one which is crawled daily by google and a newer one that was last crawled April 9th. Why would that be? I thought that once google begins to crawl it does so daily. We have added a lot of new content and it isnt being piced up due to no new crawl. What can we do? Thanks

 

Terabytes

10+ Year Member



 
Msg#: 29055 posted 2:23 pm on Apr 15, 2005 (gmt 0)

(IMO) Google crawls the site depending upon a few factors, one of which is it's knowledge of how often the site is updated.

During a recent ovehaul of my site (over a week), G just happened to index the site during updates.

I returned the next day and discovered additional updates (due to my updating...)...and crawled the site again..

My best guess it it thought that the site was being updated very frequently during that week, and ended up coming every 20 minutes or so looking for updates..

since the update, it's slacked off to once a day (or so)...however I make small updates every couple of days to keep it fresh, and entice indexing of my site.

This also holds true to web sites that are never updated, I believe G measures time/updates to compute how often to re-index the site. If you've never changed your site in 24 months, don't expect the crawler to be there anytime soon...it has millions of other sites that ARE updated regularly.

If it's a new site, you probably have no control over when, but once you see that it's indexed a page or 2 do a small amount of changing to the pages so that it at least thinks the site is kept fresh. I believe that will help in the long run.

narsticle

5+ Year Member



 
Msg#: 29055 posted 2:32 pm on Apr 15, 2005 (gmt 0)

I dont know how accurate your opinion is because our other site is crawled daily and hasnt been updated in approximately 3 years. This site newer site is being updated daily and still not getting crawled.

Brett_Tabke

WebmasterWorld Administrator brett_tabke us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 29055 posted 1:42 pm on Apr 19, 2005 (gmt 0)

put in a "if modified since" header and watch your crawls fall significantly.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved