Forum Moderators: open

Message Too Old, No Replies

Crawl and the update

Does it only crawl right after?

         

Helpmebe1

5:12 am on Aug 20, 2002 (gmt 0)

10+ Year Member



Hi all.. With the dance some predict on for the 24th or so this month.. I am hoping someone can set me clear on its patterns of deep crawling. I have been told google does the deep crawl between the 25th- 2nd of each month..

My question is, is this the only time Google deep crawls? Unfortuntately, to make a long story short- i can not see my logs to tell when he comes. Im hoping someone can answer this and let me know.. do I need to have my site updates done, say by the 28th to be safe of this month for that content that is done by the 28th to appear in the dance a month from now?

TheDave

5:22 am on Aug 20, 2002 (gmt 0)

10+ Year Member



Google visited me about a week ago, so I would say that the crawl is a never ending thing. It would seem illogical of them to try and cram a crawl into a few days, especially with that huge number of web sites it is searching.

OZZY2662

5:28 am on Aug 20, 2002 (gmt 0)

10+ Year Member



Google has been visiting me daily, 30 to 40 hits. Today, August 19th, Googlebot had over 5,000 hits on my site. The dance will start soon, August 23rd.

Helpmebe1

5:35 am on Aug 20, 2002 (gmt 0)

10+ Year Member



can we take bets when the dance will be? :)

Seriously though, you all feel this one week time span is not a realistic deep crawl? I know google does visits daily but how about deep crawls? Spaced throughout the month or does deepcrawling only in the week following the dance?

webman

5:58 pm on Aug 20, 2002 (gmt 0)

10+ Year Member



I've noticed that Google deep crawls my various sites at different times each month. I can't make out any specific trends, but here's what I see in my logs:

- My higher PR sites (>5) get deep crawled twice. Once a few days after the update and then once a week or so before the next update. These sites also get visited on a regular basis by Googlebot (about 3-4 times per week), but usually only to the homepage.
- Lower PR sites (<5) only get deep crawled once after the update, and it is usually a few days to a week after the higher PR sites are crawled.
- My guess is that everyone's crawls look different, depending on PR, the number of links to their site, the way the site is setup (is it easy to deepcrawl?) and many other factors that are beyond me. The most important factor seems to be PR.

Helpmebe1

6:03 pm on Aug 20, 2002 (gmt 0)

10+ Year Member



Webman,
Very cool..Thank you! Very well spoken, I thought PR might play a factor, the higher PR sites get crawled more often. I am hoping in the next dance to see the PR shoot up to 5 or so with some of the work we have done over the last month (this is a new site that just opened 6 weeks ago) I will try and do all my updates just incase we havent achieved that PR though and will try and have them all done by the end of the month, just to play it on the safe side...

Thanks again, I really appreciate your detailed answer!

Chris