Welcome to WebmasterWorld Guest from 54.196.231.129

Forum Moderators: open

Message Too Old, No Replies

Deep Crawl

When was your last?

     
6:56 pm on Oct 19, 2003 (gmt 0)

Full Member

10+ Year Member

joined:Feb 25, 2002
posts:311
votes: 0


I have a site with a page rank of 6, which has plenty of links "1000 plus external". I seem to have a new fresh tag on my homepage and pages 1 deep every couple of days. But pages 2 or 3 or 4 deep from the homepage seem rarely to get a crawl and it's taking weeks if not months to get new content in. Am I doing something wrong? or are plenty of other webmasters seeing this?
12:31 am on Oct 20, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:May 29, 2003
posts:166
votes: 0


It's ok, high pages are crawled often.

Deepcrawl happens between cycles of one and three weeks:
-deepcrawl
-1 week with freshbot
-deepcrawl
-3 weeks with freshbot
-deepcrawl

Last three months like this for my two sites.

1:29 am on Oct 20, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 13, 2003
posts:630
votes: 0


I get a full crawl every day. PR=5
2:33 am on Oct 20, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Nov 14, 2002
posts:422
votes: 0


Had a brand new site with no PR (2 months old with only 2 pages listed today) crawled last Wednesday (20/10). 24K pages taken up. The data is not in the 'new' index so far.. I assume its waiting in the next major update Google is going to do with new PR / SERPS being rolled out.

I get deepcrawled every 3 to 4 weeks on the PR5 sites.

I'm seeing the Image craweler on a couple of sites today (and may be over the weekend -- I did not check) on the PR5 sites.

Freshbot (or whatever its called these days) is around a few times a day on my forum sites.

4:36 am on Oct 21, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 22, 2002
posts:1807
votes: 1


How many pages is your site Plasma?

Are you sure it's not "freshbot" crawling?

6:54 am on Oct 21, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 28, 2003
posts:925
votes: 0


plasma, are you updating every page every day :)