Is the crawl done?

No hits from Deep Crawler in 4 hours.

[peterdaly]

Should I be worried? GoogleBot (deep) has not hit any of my 5 sites in about 4 hours. My major concern is Google has gotten about half the number of pages so far this crawl as last crawl.

It started very slowly this month, and didn't start getting serious until a few days into the crawl.

Am I alone in GoogleBot being gone? Is it done?

-Pete

[Alternative Future]

Nope still kicking around here. Been a little slower than in the past though!

-gs

[leifwessman]

I have the exact same experience!

[uber_boy]

After a couple of intense days during which the Googlebots crawled about 55000 pages (which is considerably less than ususal), things went calm about 4am this morning. No sign of the Deep Crawlers since that time. So if the Deep Crawl is over, it's been a much less extensive one for me than the last couple of months...

[guddu]

Hi,

I have been experiencing the same. Since, the major deep crawl on March 10th, my sites have not been deep crawled even once. The freshbot is also slow.

[Jesse_Smith]

I think I scared him away this morning!

[farside847]

I saw about 25000 hits a day for a week, but today only the freshbot. Funny thing is - I only have 60000 some odd pages so it would seem deepbot crawed my entire site more than once (although I have not confirmed that...)

[skipfactor]

Only slow freshbot since 3AM here too. Is he bringing up the rear for Deepbot? Get back to work Deepbot, you're not done.

[leifwessman]

Deepbot is back!

[uber_boy]

Yes, the Deep Crawlers have returned to my site as well. I'd be interested to know whether the Deep Crawl takes a full-on intermission or whether it simply moves on to other sites for a time. In any case, I'm pleased to have it back in the house...

[oLeon]

coukd it be that you are a bit paranoid? ;-)

itīs the end of the spider cycle, so only tomorrow and maybe the day after tomorrow the deepbot is on the way. then itīs over for a few weeks.

[peterdaly]

Back for me as well. (I started this thread.)

Anyone know this to have happened before? It is obviously more than just me, as other people saw it stop, and come back more or less on all sites at once.

Hopefully it still has time to get the other half of the pages it usuallty gets.

-Pete

[KakenBetaal]

Still here...

[why2kit]

Well wouldn't depend on how the google database is designed? and how the bots search - is it by domain name or ip address of the server, or the number of pages he wants to look at?

On my server I've seen an almost perfect bell curve starting 3/11 peaking 3/15 and decling down today 3/18

[MetropolisRobot]

IMHO DeepCrawl is not done just yet. Still some pages being spidered. It did take a break though which was noticeable.

And just to scare people. I have one site that was deep crawled back in December and has never been deep crawled again. Don't ask me why. The PRs are still fine. Freshbot appears from time to time, but deepbot is not there.

Sometimes we webmasters panic a little too readily.

[oLeon]

normally the deepcrawl is separated into to sections with a little break between them. (break doesnīt mean, that it doesnīt come at all - it only takes a very few pages during this time) the first part takes about 5 to 6 days, the second one 3 to 5.

[bignooz]

Googlebot is crawling my server since march 13th and it is still there. Donīt stop babyyy :)

[joker197cinque]

DeepBot still here.

Btw...how long a deepcrawl usually go on?

thx

fabriZio

[Jesse_Smith]

10 days

[Hefner]

i dont get why the deepcrawl doesnt hit all of the pages its supposed to hit. any simple explainations or ways to get it, maybe its beyond our control?

[kwngian]

Same there. Deepcrawl was constantly grabbing pages then it abruptly stop and does not come back.

Something wrong. Something to do with the war or what?

kwngian

[uber_boy]

Since I first posted to this thread a couple of days ago, the Deep Cralers have been coming and going with no real pattern. There have been hours where I served them more than 1000 pages and others where it's been less than 50. It's not idea but so long as it continues I'll happily endure it...