Welcome to WebmasterWorld Guest from

Forum Moderators: DixonJones & mademetop

Message Too Old, No Replies

cookies missing from a large percentage of records in log file

Large percentage of records in log file does not show cookies

7:53 am on Aug 6, 2005 (gmt 0)

5+ Year Member

We wanted to use persistent cookies on all our pages. So just to test it out, we put the code for cookie in just one page. We also have enabled asp session tracking. We imported one dayís log file into a SQL database. The results have been surprising.

Page views: 1490
Records with no cookie: 185
Records with cookie inserted by us: 185
Records with ASP session id in the cookie: 735
Records with siteserver=id=a-long-number: 5

Here are specific questions
a.We never did anything specific to get siteserver=id in our log file. We have no idea how it got there. And that too on just 5 records out of 1490. How did this happen?
b.We have enabled asp session id. What I donít understand is that there are 735 records with asp session id in the cookie, but only 185 records with our variables in the cookie.

I assume that part of the traffic would be crawlers and people who disable even first party cookies. But would that number so high?

Would appreciate some hints on where to look for a solution?

The whole idea of persistent cookies was to do better tracking

5:43 pm on Aug 6, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

All these stats have to do with only one page file's records? Not the whole log, right? There were 1490 requests for that pagefile?
7:20 pm on Aug 7, 2005 (gmt 0)

5+ Year Member

That's correct. 1490 is the page visits for just that page. All the numbers we got from running queries on the database.
2:04 pm on Aug 9, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

If it were me I would start with the fact that when a cookie is issued by a page it may not actually show up in the hit for that page. I don't have a lot of experience with sites that issue cookies sporadically like this, but I would take that day's logs, throw away lines not having to do with page files, and sort it with Access or Excel into sessions (based on a concatenation of IP and UA field plus time). I'd examine each of those visits that contained your cookied page, and see what's happening in the cookie field. You might find that your cookie is happening but not showing up until the next page view's log line. If it is, that's your answer. If on the other hand it's tending to not show up at all, while the ASPSESSION cookie is, I'd start thinking about your custom cookie's programming.
5:23 pm on Aug 9, 2005 (gmt 0)

10+ Year Member

The first request for a page that adds a cookie WILL NOT include the cookie. Think of it this way:

A user requests a page. Your server records the request in the log. The response includes a command to save a cookie in with the HTML, etc.

For the second request (usually an image on the page) the request includes the cookie (all requests include the cookie value when it exists). This cookie is dutifully recorded in the log along with all the usual data.

5:33 am on Aug 10, 2005 (gmt 0)

5+ Year Member

cgrantski and Ben, thanks so much for your inputs.

No, I didn't realize the bit about first request not including the cookie. I think both your inputs combined answers my question to some extent. Testing cookie for just one page probably was not a good idea. We will do it on a few more pages and then collect the results again. I will post the results here again

5:36 am on Aug 10, 2005 (gmt 0)

5+ Year Member

One more thing I can try. Look for all records that have this cookie page as a referrer. By this time, the records should have cookies. Right? I will do that and post the info here.
11:29 am on Aug 10, 2005 (gmt 0)

10+ Year Member

You can also try checking images on that page, which usually won't be requested until after the cookie has been set. If you want to be double-sure, you can insert javascript to request an image well after the page is loaded. The Machiavellian would request a 1 * 1 gif every ten seconds, whith a unique URL parameter on the gif, so that you can work out how long they were on the page as well. It would require custom analysis, though.
7:27 am on Aug 11, 2005 (gmt 0)

5+ Year Member

Just wanted to update based on what i learned from this thread. I decided to analyse pages that come after the page from where the cookie was inserted.

Page A: the page where we were setting cookie
Page B: pages that were referred by page A

I looked at all records for Page B and these are the results
Total pages:847
Records that have the values we explicitly inserted:610
Records that have the session id in the cookie, but not the values we inserted: 237

The good news is that around 72% visits are showing cookies correctly. What we are not able to find is why certain records have asp sessionids showing up in the [cs(cookie)] field in the log file, but not the values we are inserting in the cookie. I tried looking at client ips to see if there is one IP that is skewing the data. That's not the case. They are different IPs.

Hope I'm not confusing anyone here.