Unique visitors

how do you count them and do you trust the results?

[ppg]

Do you rely on software such as webtrends to do it for you?

How do you get round the problem of users connecting through a proxy skewing your results?

If you do it yourself presumably you put a time limit on what you'd call an average visit. How long do you allow - half an hour? more?

I see a fair few posts on the forums here where people mention the number of uniques they get, pageviews per unique etc.

I wonder if anyone has a reliable way to count them (surely impossible given the way http works?) or perhaps a way to reduce the unreliability, or do you just live with the fact that you can't trust the results - in which case, why count them at all?

At the moment I'm just counting pageviews, but I'd like to be able to count pageviews per visitor, but then I remember all the reasons I don't count uniques.

Any thoughts?

[Hannu]

We use cookies to determine unique visitors.

I never trust the numbers for unique visitors totally - I don't believe you'll ever get the true number - even with the use of cookies. There are users who don't accept cookies. And in the case of users operating system being Windows it counts "profiles" - not humans.

But then again, having numbers you can't trust totally is better than not having any numbers at all.

[ppg]

Thanks for the reply Hannu.

If you're using cookies does this mean you need to make every page on a given site dynamic rather than staight html? Wouldn't this slow down your page's load times?

what technology do you use to set/read the cookie?

[transistor]

Hello PPG, Hannu
PPG: You can use JavaScript to set cookies and, of course, you could use PHP, ASP, etc. to do the same thing.
There's no need to make the site dynamic to be able to use cookies.
Then checking how many different cookies were used gives you an idea of unique users, and I say idea, because for example, a user might visit you site from the office in the morning, then go home and check it again from his home computer: two different cookies, same user, see?
Another example would be a user visiting your site from a public library or cyber-cafe, if he returns to visit your site, even on the same computer, the cookies would have been erased and you would have two different visits and one user.

I use PHP, so I run a custom log with MySQL to register user browsing with the cookie.

I don't know if there's a way to log cookies used without a scripting language (PHP, ASP, etc.) and then obtain a report on that, is there?

[danny]

I just count unique host/agent pairs. Sure, that can underestimate users coming through a proxy and using the same version web browser, but it may also overcount when successive requests by the same user may go through different proxies (as I think happens with AOL). The result is likely to be pretty accurate (within 5%?) over short-term time-spans (say a month), and there's enough other noise (robots, etc.) to make that as much accuracy as would be meaningful anyway.

[Hannu]

what technology do you use to set/read the cookie?

WebTrends has a free plugin for MS IIS that sets the cookies server side.

[jm_uk]

I'm responding to 'transistor's comment:

"I don't know if there's a way to log cookies used without a scripting language (PHP, ASP, etc.) and then obtain a report on that, is there?"

If you are using Apache as your web server you can use their mod_usertrack module to set server-side cookies and then modify your logging directive to log the cookie values in your web server's log.

With Netscape/iPlanet, there are several openly available NSAPI cookie setting modules. Again, you will need to modify the web server's logging configuration (via obj.conf) to ensure that you log the cookie values.

With Microsoft's IIS (with which I am a lot less familiar) I believe there is openly available ISAPI code for setting of server-side cookies. With the correct configuration these cookies can then be logged in the W3C log.

Also, some web site's choose to log both the 'set-cookie' and 'cookie-header' variables in their logs. This distinguishes between your server sending a cookie value as part of a response to a request which did not send a cookie (set-cookie) and your server logging a cookie value which was sent with a request (cookie-header).

The benefit of this is that you can check your logs to see which cookies that were sent out in the 'set-cookie' field were subsequently received in the 'cookie-header' field. This will enable you to see which cookies were accepted by browsers. You can then exclude the cookies that were not accepted from your unique user analysis algorithm.

[ppg]

jm_uk and danny:

thanks for your responses. I know I can do the setting/getting with jsp which my site runs on, but I don't want to have to make all my pages jsp's.

There's no need to make the site dynamic to be able to use cookies.

for what its worth, I call any page which contains scripting (whether jsp/asp/php/perl - whatever) to generate anything apart from straight HTML for request a dynamic page.

Any suggestions that avoid making more dynamic pages sound good to me, I'll look into both of those.

[transistor]

Thank you jm_uk, that's very useful! :)

[bcc1234]

A simple way would be to use javascript to generate some unique value and store it in a cookie.
After that, place a 1x1 image on your pages and use javascript's document.write to add request parameters to that image based on the set unique value.

You'll get logs with something like:

GET /myimg.gif?JHhj43DFS
GET /myimg.gif?8hJHhj
GET /myimg.gif?fsd8832

etc...

Then calculate the number of unique parameters for that request.

That's the worst method I could think of, but does not require server-side of any kind.