Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- I want Google only to crawl my website at night


jdMorgan - 6:06 pm on Jan 27, 2010 (gmt 0)


"* c) Have your server respond with HTTP result code 503 to all users when it's bogged down"
is technically correct.

10.5.4 503 Service Unavailable

The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.

RFC2616: Hypertext Transfer Protocol -- HTTP/1.1 [w3.org] (section 10.5.4)

See that "Retry-After" header they mention? There's the direct answer to the question posed in the subject line of this thread. However, I can't comment on whether this approach is advisable, having never tried it.

Jim


Thread source:: http://www.webmasterworld.com/google/4068686.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com