Lots of 500 errors last several weeks, but why?

[dickbaker]

Out of the 25,000 to 35,000 page views I get each day, I've generally had 18-35 pages get 500 errors. On checking these out, they've mostly been dynamic pages that are still indexed by Google but the record no longer exists (example: www.somesite.com/results.asp?ID=123).

A few weeks ago I noticed that I was starting to get many more 500 errors each day. I've been getting hundreds. On a couple of days I got over 2,000.

The pages getting the errors on these days were "static" ASP pages that didn't query the database.

My hosting company thought that the errors were because I was hitting memory limits on the server, so I switched to a server with double the resources. I'm still getting an unusually high number of 500 errors.

Does anyone have an idea as to why this is happening?

Thanks for any replies.

[Ocean10000]

First step is to get more information on the type of error. Otherwise we will be guessing which will not help you much, except give you a few gray hairs.

I am suggesting you create a ErrorCapture page which will query the asp error object [msdn.microsoft.com], and have the page log it to file or email you the results.

To make this work you need to tweak IIS to map 500:100 Internal Server Errors to your capture page.

Once you have the errors details you should be able to better trouble shoot the problems.

[dickbaker]

Thanks for the reply, Ocean10000. Some of the reasons for 500 errors are obvious, such as there not being a record in the database for the one being requested.

The ones I'm most concerned about are those for my shopping cart page, for obvious reasons. Here are the error messages for the cart.asp page from Thursday and Friday:

¦154¦800a0400¦Expected_statement

¦1389¦80040e21¦Multiple-step_OLE_DB_operation_generated_errors._Check_each_OLE_DB_status_value__if_available._No_work_was_done.

¦1389¦80040e21¦Multiple-step_OLE_DB_operation_generated_errors._Check_each_OLE_DB_status_value__if_available._No_work_was_done.

¦1389¦80040e21¦Multiple-step_OLE_DB_operation_generated_errors._Check_each_OLE_DB_status_value__if_available._No_work_was_done.

¦41¦80004005¦Cannot_open_database__Myproducts__requested_by_the_login._The_login_failed.

I'm guessing that the three multiple-step errors are due to a user not checking a radio button for a product before clicking the "buy" button. The other two errors I can't figure out.

Any suggestions?

[dickbaker]

Duplicate Post - Deleted

[edited by: Ocean10000 at 3:04 pm (utc) on Nov. 9, 2009]

[Ocean10000]

¦154¦800a0400¦Expected_statement

Usually means you have a statement that split on two lines. And you forgot to do "_" to tell vbscript the rest of the statement is on the following line.

[Ocean10000]

¦41¦80004005¦Cannot_open_database__Myproducts__requested_by_the_login._The_login_failed.

It did not like the login information for some reason.

[dickbaker]

As I said, not all 500 errors are from dynamic pages. All of the 500 errors are from pages with the .asp suffix, but many of those pages have no queries in them at all, nor are they being accessed by a form or any other method than just a static link. Here's two entries from yesterday

########20:18:14W3SVC1410461454H175138{my IP}GET/productinfo.asp¦-¦ASP_0146¦New_Session_Failed80-{visitor's IP}Mozilla/5.0+(Windows;+U;+Windows+NT+6.0;+en-US;+rv:1.9.0.11)+Gecko/2009060215+Firefox/3.0.11+(.NET+CLR+3.5.30729)__utma=143293219.1727279618.1254675604.1254675604.1258316189.2;+__utmz=143293219.1258316189.2.2.utmccn=(organic)¦utmcsr=google¦utmctr=widget+stores+in+somestate¦utmcmd=organic;+ASPSESSIONIDSCRTARSD=KPPPONJDBJLPMJMMAPFBOCDJ;+__utmb=143293219;+__utmc=143293219http://www.mysite.com/somestatebytown.asp 500 0 0 223 737 46

########20:18:15W3SVC1410461454H175138{my IP}GET/productinfo.asp¦-¦ASP_0146¦New_Session_Failed80-{visitor's IP}Mozilla/5.0+(Windows;+U;+Windows+NT+6.0;+en-US;+rv:1.9.0.11)+Gecko/2009060215+Firefox/3.0.11+(.NET+CLR+3.5.30729)__utma=143293219.1727279618.1254675604.1254675604.1258316189.2;+__utmz=143293219.1258316189.2.2.utmccn=(organic)¦utmcsr=google¦utmctr=widget+stores+in+somestate¦utmcmd=organic;+ASPSESSIONIDSCRTARSD=KPPPONJDBJLPMJMMAPFBOCDJ;+__utmb=143293219;+__utmc=143293219http://www.mysite.com/somestatebytown.asp 500 0 0 223 737 62

Do static pages get 500 errors? My hosting company keeps telling me this is a developer problem, but I haven't changed anything on these pages in quite some time.

For a minute I thought it might be the banner ads that a client is running on my site, but the page with the most 500 errors doesn't have any banner ads on it. It's a dynamic page and serves up results based upon ID.

Any insights would really be appreciated.

[Ocean10000]

Here is a stupid question is session state required on those page? If not why not disable it on those pages.

I have only found one reference but not sure if this will solve your problem.
KB210842 [support.microsoft.com]

[dickbaker]

No sessions on the pages.

I've sent that link to tech support at the hosting company. Thanks for looking at this.

[dstiles]

A few suggestions, probably ones you've already thought of:

1. errors when there is no content - presumably you are trapping error returns (On Error...) when accessing databases or external modules.

2. is the path through a script different in failed cases to successful ones (if this ... else that).

3. asp does not always recognise casts (eg comparing Num1 with Num2) without both numbers (or whatever) being forcibly cast.

4. MS updates sometimes alter the way script keywords work, usually very subtly. It may not always fail and not always obviously but I have had it happen.

[dickbaker]

A guy in the tech support department at my hosting company has been looking at this. Here's his email from today:

After parsing the web logs we found that the majority of the 500 errors are "New_Session_Failed" errors. These errors occur when the application pool doesn't have enough memory to process the request. It doesn't matter whether the requested page is a dynamic script or static HTML.

You should take steps to reduce memory usage on your system by lowering the SQL Server memory limit, stopping unnecessary services, and possibly setting the applicaiton pool to recycle on a set schedule or at certain memory limits.

A few weeks ago I moved my site to a Cloud VPS server with 1GB of RAM instead of the 500 MB I have now, and more CPU capability. I had the site live for only an hour, because there were problems with the shopping cart. There were so many problems that I decided to stick with the VPS I have now for the time being.

The interesting thing is that, during that one hour, there were 81 500 errors. The increased memory and server resources didn't seem to solve the problem.

[dstiles]

I can't say I've ever seen anything like that in my logs. Sounds like scripting or SQL problems.

How many SQL accesses do you make per page?

Do you always close SQL requests after use?

How many SQL channels do you keep open and do you re-use them?

How much do you dump into session or application variables?

Do you have any recursive calls to functions? If so do they always exit and are they limited in depth?

[dickbaker]

The databases are on a server separate from the VPS my site resides on.

I don't know what you mean by "accesses." If you mean the number of queries per page, it would be one. The page is selecting various fields from the database table based upon one or two variables.

The requests are closed after use.

The site doesn't use session variables often. Most requests are through form variables or URL requests.

I don't know what recursive calls are. I just searched for the term, and it seems to mean complex queries with unions or joins. The queries I have a pretty straightforward: select A,B, C from dbo.table where Something = 'A' and SomethingElse = 'B'

What I don't understand is why I didn't see this number of 500 errors last year at this time when my traffic was 30-50% higher.

[dstiles]

Sorry, just some random ideas I thought might be relevant. :)

Could it be delays between your site and the remote SQL databases? How many queries are opened simultaneously (eg how many pages at a time) and are all requests being honored by the db server. It may be that the sessions are being held open for far too long because of incompletion - although I assume there is a timeout for the transaction with suitable action on failure.

Recursion is when a function calls itself (and perhaps calls itself again, and again...): ie a single useful function (perhaps a string conversion process) can be called many times by itself in order to complete a sequence of similar actions. If you didn't know that I assume you aren't using recursion. :)

If the errors weren't there last year then you obviously have to evaluate what changed. I don't envy you that task. :(

[Seb7]

Check your 'AspSessionMax' value in the IIS meta file.