Deep Crawl/Site Down

[jen24815]

Hi Guys,

My site was down for 8 hours today (I've since purchased some back-up "insurance" in case this happens again).

Here's the deal:

The deep crawler (216.239....) hit my robots.txt and my index page about 10 mins. before I crashed...

Will I still be in the index next update? Is it going to drop any of my other pages that didn't get hit today?

Freshbot crawls me every day, but I know all my interior pages missed the deep crawl (at least for today) and only my main page got hit.

Any opinions?

Thanks!

Jenny

[percentages]

Jenny,

To me Google seems fairly tollerant on pages that are down at a particular instance. It may not get the updates, but it doesn't seem to drop pages simply because it couldn't get to them on the first attempt.

Reliable hosting is however invaluable. Suffer this fait a couple of times in a row and you maybe in trouble.

However, for a one time problem I don't think Google will care :)

[JayC]

It's also not completely certain that you've missed the deep crawl, in spite of having seen that robots.txt/index.html hit. For example, the site I'm watching most closely this update got visited with that same request last night sometime before 10 pm (when the logs rolled over), but the first deep crawl hits began just a few hours ago -- more than 24 hours later.

While there's no way to predict what might happen, I wouldn't panic yet.

[Lots0]

I am going here on my own past experiences -
First has googlebot or freshbot hit your site after it came back up? I have often seen googlebot come back within 24 hours to re-check a site durring the deep crawl. (I’m assuming that you have some good links pointing to your site.)

If you don’t see googlebot or the newer freshbot after a week or so you might possibly be out of the next index.