Forum Moderators: open

Message Too Old, No Replies

Random ASP0115 error

         

dstiles

10:20 pm on Apr 4, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



This has been driving me nuts for about a year. It's now imperative that I fix it.

Mostly well-established code on established web sites running for at least two or three years.

There is no error on a local Windows 2000 dev server

There was no error on a previous 2003/IIS6 server.

About a year ago I transferred every site to a new 2003/IIS6 server (different datacentre so possibly different OS build). Four or five sites immediatley gave the 0115 but very infrequently - some sites maybe once a week, four or five times a day on the most-accessed sites.

I tried several things and went through all I could find about the error online. I even tried running the IIS debug software: the output made no sense. I separated the "broken" sites into their own App Pools, no change. Hosting company has no idea. I ran out of time and decided that the errors were sufficiently infrequent as to be ignorable.

Most of the affected sites access an MDB database (different one for each site); one (site SD) uses MySql. They use the same library code as several dozen other sites that also access MDB's and MySql's with no problem, even though some other sites have higher access rates.

I have now added a new layer of software to a few test sites, notably one that previously had this problem (site SD using MySql) and two high-access sites that didn't (sites AB and MR using MDB) ("high-access" is a relative term - they are all fairly low by some standards).

This new layer, a replacement for my aging security traps, accesses a MySql database before anything on the actual sites gets parsed. The MySql access replaces a text file read on a previous version of the traps (which are still running error-free on many other sites). All the new Trap sites now exhibit the 0115 problem, again infrequently but more frequently on two of the high-access sites, probably because there are more pages to react.

Another "site" is used as a Control Panel to read/write the MySql database and that failed as much as any of the sites. I thought I'd fixed the problem in this CP but succeeded only in reducing the incidence; however this did suggest a possible reason: excess createobject calls.

One of the instances (MR) has a higher failure rate BUT the page that features most comprises (apart from the MySql security hit) only a single read of a text file (using createobject). That's just two Creates in a single page of minimal external objects.

The faults seem to occur at times I would expect high-ish activity on the server on AB and MR sites.

The CP failure seems to happen only at times when other sites throw up an error (ie when the server is busy), and yet it comprises a single page with a single Create and half a dozen reads of a single table at a rate of about one page per 30 seconds tops (I'm requesting the page through a browser). The latest failure of this CP page, which I'm able to run with a lot of tracing lines, shows the failure happens on the first Create/Open (ADODB.Connection and dataconn.Open). I have seen failures at other times later in the sequence than this.

What it is not: missing form names (this is a known 0115 report): most affected pages are not forms driven and ALL fields have names.

No database field has a Reserved name (nor do the forms).

It may be session names (another known 0115 report) but I doubt it: the MR text-read page has only a handful of them.

None of the other MS excuses seem to be relevant.

My impression is that there are too many objects created at certain times by the SERVER (I say this because failures include the minimalist CP page).

My GUESS is that at peak times the number of Creates (and their allied processing) has been increased by the extra Trap script beyond a boundary quantity for the SERVER (because it's using MySql not flat file).

Is this possible?

Is there a way to increase the number of objects available if it is?

Does anyone out there have any helpful ideas on this?

Please? :(

Staffa

10:17 am on Apr 5, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Just a wild guess here but do you have all your createobject calls set to nothing when they are no longer needed ?

As I say, just a wild guess.

dstiles

2:30 pm on Apr 5, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



As far as I can tell, I've been through them several times. Unless they accumulate AFTER a script has been finished I don't think the one that fails on the first ever opening in a script would happen.

Thanks for the suggestion, though. :)

Staffa

4:21 pm on Apr 5, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Another wild guess; if it fails on the first opening, could there be something in your global.asa file, that is if you use such file.

dstiles

10:57 pm on Apr 5, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Sorry, still no cigar: no global.asa. :)

Nearest I've got are a couple of Application Vars but those have been running forever - well, apart from being re-primed after every bug-fix reboot. :)

There were 31 failures today - by far the most so far (13 on Sunday). Expected, in a way, since two of the test sites are genealogy-ish oriented which means people on a wet day off, and it was mostly those sites that failed.

Staffa

11:29 pm on Apr 5, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'm running out of guesses :)
Just one more. As I understand it, asp executes a file line by line, top to bottom so if you would have for example
a CreateObject
then whatever needs to be done
then a response.Redirect
and then close, set to nothing

the visitor would have moved on before the close/set to nothing is executed which might leave them hanging.

Again just a guess, I'm sure someone better qualified here may have the answer.

dstiles

10:59 pm on Apr 6, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I see what you're saying, but almost all of my code is in functions and subs so the punter wouldn't even begin to get a page before I'd processed the data. I think.

I took your suggestion about closing objects and had another look at it. All objects were set to Nothing but a couple of opened objects weren't closed (eg rs.Close). They are now, although there's something causing a problem I have to look into. Unless I keep the objects open throughout the trap sequence, instead of closing one object and opening another (eg open, save, read, close rather than open, save, close, open read, close), I get errors when closing the interim ones, despite them not having been explicitly closed. I need to look further into that but the traps software is working ok by not closing interims: it's weird.

25 errors today.

Thanks for your input, anyway. It's making me think in different ways about what's happening.

dstiles

8:11 pm on Apr 8, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I've made a few more changes which I'll list here, although I don't think this investigation is going the right way.

Changed from DSN to DSN-Less on Traps accesses. No difference. Since it's harder to maintain and isn't ODBC-less I will later revert to DSN connection, which is supposed to be slower but I don't have to expose the password.

I tried to update to the latest version of MyODBC (V5.1.6 from 1.5.5) but since I would have had to bring the server offline and lose all sites for 15 minutes or so I didn't follow through.

With the DSN-less connection still in place, I have added indexes to the two fields which carry the bulk of connections. Hopefully this will reduce the number of disk accesses (although likely to slow down access at various points). I do not expect this to fix the basic 0115 problem since that has been observed at the conn.Open point, prior to any actual SQL request being made.

I'm about to go out to my local agricultural supplier and buy some more straws to clutch at. :(

The basic problem does seem to be too many Objects being created (or at least subsequently being Opened) with CreateObject. This idea is encouraged by the failure rate being higher when there is a lot of traffic. Does anyone have any idea how to extend the number of available Objects?

To counter that, only a very few sites fail at all: those original ones noted in the OP and those to which the Traps has been added. Other MDB-using sites do not seem to be affected, although possibly timing could affect that (ie not opened in peak traffic periods).

Ocean10000

8:30 pm on Apr 8, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Since this is most often happening in a high traffic period have you tried looking in the temp directory. This might have to do with temporary file creation/management. I am assuming the objects you are trying to create will use a temporary file for some reason.

I am guessing the temporary file name was issued twice in a short span of time. And on the second attempt, the file was already in use and the object creation failed. And this failed creation error bubbled up and morphed into what you are seeing.

dstiles

12:24 am on Apr 9, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Interesting suggestion!

I am fairly certain it's not a MySql error since there are no errors in the log, so that leaves IIS sessions (possible but unlikely, I think), or ADO, which I would have thought might be held in memory?

There are a couple of places that I know of where temp files are created:

C:\temp - Only MDB Jet in that folder at the moment. I don't think that is the problem anyway. (Most MDBs are DSN-less.)

C:\WINDOWS\system32\inetsrv\ASP Compiled Templates - IIS sessions: there are 500 memory sessions available and 500 disk sessions; I was surprised to see the latter used at all but it currently has half a dozen). Again I don't think that's it UNLESS the ADO info is stored in those sessions - I suppose it may well be. Any idea if it is?

A problem in checking this, of course, is that the failure points are of fairly short duration: anything from one off to a couple in ten minutes. Without watching at the specific failure time (which means ALL the time) this is going to be difficult to spot. I'll try to work out the recurrence pattern and keep refreshing the Application Event list around that time. :(

If you are correct, though, what does that say about MS's temp filename management? :(

But thinking about it: the machine is far faster than the old ones, which never failed; it could be reacting too fast, I suppose.

Ocean10000

6:36 pm on Apr 9, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



GetTempFileName [msdn.microsoft.com] I think this is the API call which most application use to create a temporary file, even dot.net.

Only the lower 16 bits of the uUnique parameter are used. This limits GetTempFileName to a maximum of 65,535 unique file names if the lpPathName and lpPrefixString parameters remain the same.


I am only speculating this is the problem, or maybe helping cause your problem.

dstiles

10:44 pm on Apr 9, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hmm. I don't think it is. In the most recent case there was almost no TCP activity, according to TCPView, but the page failed on the Open function.

The failed page was not in the disk cache folder, although three other pages were cached to disk at about the same time (files are successively numbered in a single folder). This time the event was not even recorded in the Events log. If that is happening elsewhere I have no idea what the real failure rate is without grepping a large number of big logs. :(

I have no idea what the cache timeout is so not sure if it should even have been writing to disk. As noted earlier, there are 500 memory caches and 500 disk caches. Early in the investigation I enabled disk caching (it was set to Cache but the quantity was set to zero - as far as I know this was a default MS setting). I have now turned off disk cache properly.

The number of script engines to cache was 300: I've just upgraded it to 500.

One change I made last night was to paste the contents of two include files into Traps script, which previously loaded those two files when required (always). This made no difference so to simplify maintenance I will revert to the inclusion method.

I'm about to go through one of the scripts again - the latest failed script is very small, being a database "control panel" for the Traps. The DB part of the script is very simple:

Create ADOB.Connection
Open Connection (which seems to be the failure point)
Create Recordset with SQL string
If successful, read/process records (failures always have records availabe)
Close Recordset
Close Connection

Sometimes there is a save or delete before the Read but that's the gist of it. And since the failures seem to be on the Open (the ones I've seen, anyway) it seems immaterial what happens afterwards.

Note: In an early stage of the investigation there were separate create/opens (with intervening closures) for the Save/Delete phase and the Read phase. The connection is now held open for the complete process. In some cases I noticed a failure on the second (Read) phase rather than on the Save/Delete phase but it seems to have been on the second Open that the failure occurred. The number of failures ON THAT SCRIPT are far fewer now but that's probably statistics at work.

It's just occurred to me to check the number of Handles and their behaviour but that looks normal - a reasonable number (42,600) increasing and decreasing with the number of threads (I've seen instances where the number of handles just kept going up!).

dstiles

10:15 pm on Apr 15, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Looks like it MAY be fixed.

I took the opportunity of this week's MS bug fix, which I knew was going to need a reboot, to install the new MyODBC that I downloaded earlier (update from 5.1.5 to 5.1.6 - messed that up in an earlier post!). That was 49 hours ago. There has been no new failure in that period. There also seem to be fewer "Aborted connection ... reading communication packets", although still a few (not worried about those).

Strangely, the two sites that do not normally use ODBC also seem to be cured. The sites DO use it but only on order submission pages, and that is not where the errors were coming from. Have to look into that one. :(

The only thing I have to do now is find out why I've been locked out of MySQL itself. SQL-Administrator, SQL-Query and even command line won't let me into a couple of "root" accounts and the backup account lets me in but doesn't let the backup scheduler in. The fourth account is for mail and that seems to be working AND lets me in, although I can't actually see the tables once I'm in. Looks like I'm going to have to stop the thing again and try to fight my way back in. :(

Thanks for the help, guys. It drove me to look in unusual places. :)

Staffa

6:00 am on Apr 16, 2010 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Good for you ! :)