I have been recently analyzing various access logs and can see many entries where Googlebot is downloading the same page twice in succession. The following is an example showing two downloads of [at-autos.example.com...]
My understanding is that the number immediately following the HTTP response code (200 in both lines above) is the number of bytes downloaded. For each pair of downloads, the first number is always (as far as I've checked) smaller than the second.
Does this imply that the first request is being interrupted and the subsequent attempt probably isn't?
Any other ideas what might be going on?
Thanks in anticipation for your help.
[edited by: bill at 9:24 am (utc) on Oct. 2, 2009] [edit reason] Use example.com [/edit]