Welcome to WebmasterWorld Guest from 23.22.46.195

Forum Moderators: open

Deep Crawl

When was your last?

   
6:56 pm on Oct 19, 2003 (gmt 0)

10+ Year Member



I have a site with a page rank of 6, which has plenty of links "1000 plus external". I seem to have a new fresh tag on my homepage and pages 1 deep every couple of days. But pages 2 or 3 or 4 deep from the homepage seem rarely to get a crawl and it's taking weeks if not months to get new content in. Am I doing something wrong? or are plenty of other webmasters seeing this?
12:31 am on Oct 20, 2003 (gmt 0)

10+ Year Member



It's ok, high pages are crawled often.

Deepcrawl happens between cycles of one and three weeks:
-deepcrawl
-1 week with freshbot
-deepcrawl
-3 weeks with freshbot
-deepcrawl

Last three months like this for my two sites.

1:29 am on Oct 20, 2003 (gmt 0)

10+ Year Member



I get a full crawl every day. PR=5
2:33 am on Oct 20, 2003 (gmt 0)

10+ Year Member



Had a brand new site with no PR (2 months old with only 2 pages listed today) crawled last Wednesday (20/10). 24K pages taken up. The data is not in the 'new' index so far.. I assume its waiting in the next major update Google is going to do with new PR / SERPS being rolled out.

I get deepcrawled every 3 to 4 weeks on the PR5 sites.

I'm seeing the Image craweler on a couple of sites today (and may be over the weekend -- I did not check) on the PR5 sites.

Freshbot (or whatever its called these days) is around a few times a day on my forum sites.

4:36 am on Oct 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



How many pages is your site Plasma?

Are you sure it's not "freshbot" crawling?

6:54 am on Oct 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



plasma, are you updating every page every day :)
 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month