Welcome to WebmasterWorld Guest from 54.167.0.111

Forum Moderators: open

Message Too Old, No Replies

Deep Crawl

When was your last?

     

mikeD

6:56 pm on Oct 19, 2003 (gmt 0)

10+ Year Member



I have a site with a page rank of 6, which has plenty of links "1000 plus external". I seem to have a new fresh tag on my homepage and pages 1 deep every couple of days. But pages 2 or 3 or 4 deep from the homepage seem rarely to get a crawl and it's taking weeks if not months to get new content in. Am I doing something wrong? or are plenty of other webmasters seeing this?

Gus_R

12:31 am on Oct 20, 2003 (gmt 0)

10+ Year Member



It's ok, high pages are crawled often.

Deepcrawl happens between cycles of one and three weeks:
-deepcrawl
-1 week with freshbot
-deepcrawl
-3 weeks with freshbot
-deepcrawl

Last three months like this for my two sites.

plasma

1:29 am on Oct 20, 2003 (gmt 0)

10+ Year Member



I get a full crawl every day. PR=5

shrirch

2:33 am on Oct 20, 2003 (gmt 0)

10+ Year Member



Had a brand new site with no PR (2 months old with only 2 pages listed today) crawled last Wednesday (20/10). 24K pages taken up. The data is not in the 'new' index so far.. I assume its waiting in the next major update Google is going to do with new PR / SERPS being rolled out.

I get deepcrawled every 3 to 4 weeks on the PR5 sites.

I'm seeing the Image craweler on a couple of sites today (and may be over the weekend -- I did not check) on the PR5 sites.

Freshbot (or whatever its called these days) is around a few times a day on my forum sites.

stuntdubl

4:36 am on Oct 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



How many pages is your site Plasma?

Are you sure it's not "freshbot" crawling?

dirkz

6:54 am on Oct 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



plasma, are you updating every page every day :)