---- What exactly do those WMT "Crawl errors" mean?
1script - 5:56 pm on Aug 13, 2011 (gmt 0)
I posted before [webmasterworld.com] about a sudden influx of 404 errors in WMT. But, as I dig further into this, I'm finding that these 404s may only be a part of my problem so I decided to start a new thread and look at this at a different angle.
Before I get to the point, I wanted to add another bit of info: I'm talking about a site that had lost 50% of its Google referrals. The troubles started on or around July 26th, so I'm not too certain if I can pin it on Panda - seems a couple of days off.
So, I'm seeing a considerable amount of other types of errors as well that makes me thing that there's something going on on the technical level (before I blame it all on Panda and move on as people here suggested :)
Here is the list of errors. Some are rather self-explanatory but some are ambiguous, so I would appreciate it you can fill in based on your previous experience:
404 (Not found) which I hear could mean not only the obvious 404 but also 410 and even 301. Does not sound like a big stretch to me to think that anything in the 400s (except 403) would fall (totally inappropriately if you ask me) into this category
Connection refused - would this be a 403? I can't confirm because I don't see a hit by Googlebot on the date shown on the URL reported with a 403 status code. There are normally just no hits on that URL at all. Where they get "refused" in this case remains a mystery to me
No response - server down? Strangely, it shows dates when I know for certain the server was up 100%
Failed to connect - physical node up but Apache is down? Just a speculation, otherwise I see no difference between it and "no response". Again, the date shown is the day I know it was up 100% of the time
Network unreachable - DNS and possibly the provider's routing errors and such? This makes sense because I'm finding out that one of my nameservers has been down for a l-o-o-ong time (quite possibly over a year) and I think it can cause DNS problems intermittently if the whole server is only supported by one nameserver.
Redirect error - usually cyclic redirects. All instances of this error seem to be reported correctly, so no quibbles here.
500 error - usually a programming error. Seems to be reported correctly all the time.
So, that's the list. I see A LOT of 404, connection refused, failed to connect, network unreachable errors, otherwise wouldn't have started this thread. So I can't just dismiss it as a fluke. Also, date reporting in WMT seems to be out of whack - shows no errors on the days I know server has issues and errors on the day I was watching it all the time and saw nothing wrong.
Other than perhaps 500 and Redirect errors, there seems to be quite a bit of ambiguity about what the errors actually mean, so I would appreciate if people can fill in better descriptions if you had to deal with it before.
P.S. I did try WMT Help first. Google employees' responses are usually as ambiguous as the error details in WMT ...