Hi guys, for a while now one of my clients sites has been returning loads of different server errors. There is no consistency to these errors and sometimes the site is ok for months. The site itself isnt complicated. It's built in asp and only has a small Access DB containing some news stories. It also has a wordpress blog sitting in its own sub folder but this doesnt not interfere with the main site.
Yesterday the site dropped 20+ places in the Google rankings which prompted me to check over things. When i navigated through the site i got a lot of 401/407 errors which went after some refreshing. I also received an email from freewebmonitoring.com who reported that they had seen a 401 error as well. Google Webmaster Tools has also reported a 401/407 error for a few pages - which explains the drop in rankings.
In the past i've made loads of changes to the site to try and narrow down the possible causes of the errors. I've changed the hosting, i've removed the MSAccess driven news story feed from the site template (so the only place that could cause an error is on the news page) and i've also spoken to the new host provider (with no resolution).
This time the host provider is saying that they dont see any 401/407 errors in the server logs and neither do i! But i witnessed the errors in my browser that day!
Im stumped and as im usually a "know it all" this is really pissing me off! Any help or if you want the URL please let me know.
Well, you can narrow it down to an authentication problem:
401 Unauthorized The request requires user authentication. The response MUST include a WWW-Authenticate header field containing a challenge applicable to the requested resource. The client MAY repeat the request with a suitable Authorization header field. If the request already included Authorization credentials, then the 401 response indicates that authorization has been refused for those credentials. If the 401 response contains the same challenge as the prior response, and the user agent has already attempted authentication at least once, then the user SHOULD be presented the entity that was given in the response, since that entity MAY include relevant diagnostic information. - 407 Proxy Authentication Required This code is similar to 401 (Unauthorized), but indicates that the client MUST first authenticate itself with the proxy. The proxy MUST return a Proxy-Authenticate header field containing a challenge applicable to the proxy for the requested resource. The client MAY repeat the request with a suitable Proxy-Authorization header field.
Now what 'agents' (pages, scripts, services) on your site might require authentication and also might be broken or might be invoked unexpectedly? (and what 'agents' might invoke them?) What URLs and filepaths are these errors associated with? -- Any particular 'kind' of URL? -- Any particular subset of directories?
Really, it seems you've a lot more information collecting to do here, as aside from the clues above, "it could be anything, really" -- Bad script, hack, etc. Also, make sure your server's not configured as an open proxy.
Msg#: 3960331 posted 9:18 am on Jul 28, 2009 (gmt 0)
Cheers JD, The sites pretty basic to be honest. Its just a standard ASP site. There's a wordpress blog but thats in a sub folder and doesnt "feed" into the main site anywhere? The only other script is a js one for the menu.
I can pm the url if you want but im pretty sure there is nothing that could be deemed as unauthorized. The errors were happening randomly on every page of the site and they'd go after a couple of refreshes. It was happening even on the simplest of pages.
Ive had some feedback from another forum that suggested the problem may be related to the server setup. Ill quote...
"Judging by the 401 and 407 error codes that is a problem with ISA server or the host is old and still using the proxy set up. The errors wont appear in the log files which you and the host provider examined because they arent getting that far. Those errors are occurring at the host firewall."
"The only explanation for seeing 401/407 errors when there are no pages, scripts or services on the site which require authentication is that the host has problems with ISA."