the next google crawl. A day or two away?

what are your thoughts?

[tankman]

I am curious if we should be expecting the crawl in a day or two? What are your thoughts?

[amoore]

Google started deeply crawling my site yesterday. (that's Sunday the 26th.) It crawled a couple hundred pages then and is on to some of my other sites now, too. Unfortunately, it hasn't been sniffing around my new site much yet.

[yetanotheruser]

I mentioned this on another thread.. GBot starting crawling one of my new sites again yesterday, (but looking at the number of unique pages spidered rather than number of total hits, he seems to be double checking still)

Thing of interest was that I was starting to get hits from people (due to the dance - it's this sites first time in google) but the hits coincided exactly with hits from the bot.

Does this just sound like coincidence or a little fascinating behaviour?

ATB :)

[annej]

I was hoping to make a few changes before the deep crawl. Is it likely too late?

Anne

[Stefan]

Maybe just coincidence, yetanotheruser... it sounds a bit too quick. You had new pages showing from the update at the same time? That would help. My traffic went way up since the changes started bouncing around the databases yesterday.

I've just had a very few freshbot hits since the start of the dance. The update is still on though, and last time the deepcrawl didn't begin until a few days after the update start, so I'm hoping for it tomorrow or the next day.

I've been madly fixing code and changing things yesterday and today... I get paranoid whenever I change a lot of things at once... my fingers are crossed.

[bether2]

This afternoon, the freshbot (64.68.82.65, etc.) just finished a three-day and unusually deep crawl of my site.

Expecting the REAL deepcrawling bot (216.239.46.42, etc.) in the next day or so. Unless the freshbot was doing double duty. :)

[taxpod]

IMHO:

There always unusual crawling activity during the dance. The real deepcrawl seldom starts for me before the first of the month.

Typically I see the real deal commence around the third and go on for 3 days. Typically there is an approx 48 period during which 50K pages are pulled down. Usually she gets hold of me, view like 5K pages and then really crawls for the 48 hours slowing down near the end.

[jtbell]

On my server, the January deep crawl began on the 3rd and ended on the 12th. This includes all the sites on the server, not just my own pages.

[astoller]

How do you folk know exactly the antics of google and how long she takes to crawl you and when she comes back etc.?
And how do you know if she pulled down 5k of pages or 50k?

[Stefan]

I have a couple of freeware log analyzers, astoller, but because I only have one site with about 50 pages, and it only gets 30-40 hits a day, (other than bots), I usually just download the raw log files and look at them in Wordpad. You can see all the details of the traffic including most bot indentifications. Google freshbot is always 64.68.xx.xx, deepcrawl bot is 216.239.xx.xx.