homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

Deep Crawl
When was your last?

 6:56 pm on Oct 19, 2003 (gmt 0)

I have a site with a page rank of 6, which has plenty of links "1000 plus external". I seem to have a new fresh tag on my homepage and pages 1 deep every couple of days. But pages 2 or 3 or 4 deep from the homepage seem rarely to get a crawl and it's taking weeks if not months to get new content in. Am I doing something wrong? or are plenty of other webmasters seeing this?



 12:31 am on Oct 20, 2003 (gmt 0)

It's ok, high pages are crawled often.

Deepcrawl happens between cycles of one and three weeks:
-1 week with freshbot
-3 weeks with freshbot

Last three months like this for my two sites.


 1:29 am on Oct 20, 2003 (gmt 0)

I get a full crawl every day. PR=5


 2:33 am on Oct 20, 2003 (gmt 0)

Had a brand new site with no PR (2 months old with only 2 pages listed today) crawled last Wednesday (20/10). 24K pages taken up. The data is not in the 'new' index so far.. I assume its waiting in the next major update Google is going to do with new PR / SERPS being rolled out.

I get deepcrawled every 3 to 4 weeks on the PR5 sites.

I'm seeing the Image craweler on a couple of sites today (and may be over the weekend -- I did not check) on the PR5 sites.

Freshbot (or whatever its called these days) is around a few times a day on my forum sites.


 4:36 am on Oct 21, 2003 (gmt 0)

How many pages is your site Plasma?

Are you sure it's not "freshbot" crawling?


 6:54 am on Oct 21, 2003 (gmt 0)

plasma, are you updating every page every day :)

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved