homepage Welcome to WebmasterWorld Guest from 54.196.62.23
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

    
Building a stats system
Information that is essential in any analytics solution
corrideat

5+ Year Member



 
Msg#: 3945327 posted 3:30 am on Jul 3, 2009 (gmt 0)

I am (in the first stages of) developing some Analytics system. The original idea is to try it on the sites of my own, even though later it will become publicly available.

It would collect the following data (using ECMAScript):
* Flash version
* Google Gears version
* Screen resolution an color depth
* UA-related information (UA string, version, OS, platform, etc.)
* Plugins installed (Mozilla-based)
* Referrer URI
* Whether the browser is in Quriks or Standard mode
* Local time and timezone information
* Support of ActiveX and XHR.
* Result of some basic DOM tests to check DOM support.

In addition, the use of server logs would provide with:
* IP address
* country of origin
* methods to check whether the user is a new or a recurring visit.

I would like to know what you think about such a system. I'd be glad if you could tell me what kind of information is actually essential to you, or that you would like to be able to know and your current system doesn't provide.

Thanks in advance

 

toplisek

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3945327 posted 7:11 am on Jul 3, 2009 (gmt 0)

What is ECMAScript?

corrideat

5+ Year Member



 
Msg#: 3945327 posted 1:48 pm on Jul 3, 2009 (gmt 0)

ECMAScript [en.wikipedia.org] is a language, formely known as JavaScript, after Netscape submitted it for standardization to Ecma International.

JavaScript (and JScript) are currently aimed to be compatible with ECMAScript although they may provide additional features that are not defined in the ECMAScript specification per se.

toplisek

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3945327 posted 2:22 pm on Jul 3, 2009 (gmt 0)

Why you would use Google Gears version in stat. application?

corrideat

5+ Year Member



 
Msg#: 3945327 posted 4:58 pm on Jul 3, 2009 (gmt 0)

toplisek, well, Google Gears provides webmasters with useful tools, and they might be interested in knowing whether their users do support it or not. The same applies for Java and Flash, albeit Google Gears is currently not so popular among users (and therefore not so popular among web developers.)
For that same reason, I am planning some test to check for Silverlight.

toplisek

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3945327 posted 7:18 pm on Jul 4, 2009 (gmt 0)

What you would like to track in color depth?

corrideat

5+ Year Member



 
Msg#: 3945327 posted 7:23 pm on Jul 4, 2009 (gmt 0)

Actually, color depth is a highly valuable data. Imagine that you find out that 20% of your visitors only support up to 256 colors. Then, you'd better provide a low-color version of the site.
Nowadays, however, it is quite probable that all your visitors will support at least 16-bit, but the data comes always handy.

janharders

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3945327 posted 7:33 pm on Jul 4, 2009 (gmt 0)

imho the "what do my clients support" is a) easy to find out and b) less interesting. to really bring something new into the analytics field, you should focus on "what do my clients do/want", clickstream analysis, heat maps, information on wether they scroll down on a page, time spent on page etc pp.
that'd be something new in the world of (free?) analytics.

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 3945327 posted 7:37 pm on Jul 4, 2009 (gmt 0)

Actually, color depth is a highly valuable data. Imagine that you find out that 20% of your visitors only support up to 256 colors.

If you find out your users only support 256 colors you're either running the internet in the early 90s or you're in the Twilight Zone.

The size of the display device is actually more interesting these days than the color depth.

janharders

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3945327 posted 7:42 pm on Jul 4, 2009 (gmt 0)

If you find out your users only support 256 colors you're either running the internet in the early 90s or you're in the Twilight Zone.

does anyone know how the average user in the developing countries is equipped?

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 3945327 posted 8:36 pm on Jul 4, 2009 (gmt 0)

does anyone know how the average user in the developing countries is equipped

No clue but everything would be kind of useless to them if they only had 256 colors.

I would assume smaller monitors such as 1024x768 but high color.

Think about it, with low colors any photography, video or flash content would be intolerable to watch which would make the whole internet experience less that fun.

toplisek

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3945327 posted 7:06 am on Jul 6, 2009 (gmt 0)

Why would you track Plugins installed? Are there any special?

cgrantski

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3945327 posted 12:07 pm on Jul 6, 2009 (gmt 0)

Hi Corrideat,

I can see why you'd want to track all of this if you are in charge of a web site and you want to be able to apply the latest apps and such, but you want to be sure your visitors actually can use them. And if you're selling/making plug-ins or something similar, you'd like to see what plug-ins are already being used, and watch the trends.

And I know for a fact that detecting some of the things you're after will happen only if you get to know those things extremely well, to learn the clues that give them away (some are very simple, others not so much). That's a valuable exercise to go through.

And if you don't want to install or pay for an existing analytics program, or if you feel they will be exploiting you, then writing your own program will have its own kind of payoff.

At the same time I hope you are aware that you are probably re-inventing a wheel that has been invented many times, because a lot of analytics data collection programs collect most of what you're describing, and so do simple server logs.

Also, you are talking about only the data collection piece of analytics. The further parts of analytics (sessionizing, organizing, presentation, and actual analyzing) are where you'll fall short unless you have a very thorough understanding of what all the other tools do and why. Your data will probably end up being "analyzed" by something like Excel or LogParser, and even then won't be able to be useful unless the data is summarized into visits (not hits).

That said, if you can write bits of code for detecting some of these more obscure things, and if your bits of code can be fit into other programs' analytics page tagging so they can add to what's already being collected by them, you'll do some people a service if you decide to share. It'll be difficult to add to what Google Analytics offers because Google Analytics wouldn't be very adaptable about the additional data types (prove me wrong on that and I'll be happy!), but the cost-based, more sophisticated programs like Omniture, Coremetrics, and especially WebTrends (which is extremely adapatable to additional collected information) are those that will allow their users to see the additional information in their reports. I'm not sure about Yahoo Analytics, I haven't started playing with it yet.

I realize that I haven't answered your original question at all, that is, what facts about the visitor's browser and system are important to collect. Sounds like you have a handle on that with your own ideas. If you can collect it, do it, as long as you understand that most of your list is already being done by most existing analytics programs. But somebody somewhere will need to know about Google Gears support and so on!

toplisek

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3945327 posted 12:19 pm on Jul 6, 2009 (gmt 0)

What do you mean by Referrer URI? It can be many options...

corrideat

5+ Year Member



 
Msg#: 3945327 posted 3:26 pm on Jul 6, 2009 (gmt 0)

By referrer URI I mean the URI that referred to the website. In other words, the site in which the visitor was before coming. Normally, an user agent would report it to the server using the "Referer" HTTP header.

BradleyT

10+ Year Member



 
Msg#: 3945327 posted 6:29 pm on Jul 6, 2009 (gmt 0)

Isn't the goal of studying web site analytics to improve your site's goals? If so, how is this crap going to help do that?

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 3945327 posted 6:40 pm on Jul 6, 2009 (gmt 0)

Isn't the goal of studying web site analytics to improve your site's goals? If so, how is this crap going to help do that?

Think about it and you'll answer your own question there.

From a web developer perspective having statistics and trends on customer browser capabilities helps you design for the masses, ignore fading trends and on the flip side embrace upcoming trends.

As a matter of fact the Browser Capabilities Project [browsers.garykeith.com] provides a lot of information already that many websites use to target capabilities to the devices that connect to their sites.

A couple of of WebmasterWorld members already maintain a lot of these stats so the OP may want to work with them on this project.

GaryK

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3945327 posted 8:16 pm on Jul 6, 2009 (gmt 0)

I don't know if the OP is on a Windows server. If yes then there's a commercial product that's been available for many years that does almost exactly what he wants.

My browser capabilities project is designed more to let developers know what browser-based features a browser is capable of supporting.

This commercial product provides a lot more information and lets developers know what actually is supported in both the browser and the underlying system.

[edited by: GaryK at 8:23 pm (utc) on July 6, 2009]

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 3945327 posted 11:49 pm on Jul 6, 2009 (gmt 0)

Gary, if it's the same commercial product they also have a SaaS (software as a service) version where you just embed the javascript and they track the details for you, kind of like Google Analytics, which would allow Linux webmasters to at least collect the stats.

GaryK

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3945327 posted 3:12 pm on Jul 7, 2009 (gmt 0)

Bill, I just went to their site to have a look and they have indeed added their product as a service. So whether the OP is on Linux or Windows it seems like he's all set.

BradleyT

10+ Year Member



 
Msg#: 3945327 posted 3:34 pm on Jul 7, 2009 (gmt 0)

Think about it and you'll answer your own question there.

I'm just saying I'd rather analyze user behavior rather than user capabilities since the OP asked -
I'd be glad if you could tell me what kind of information is actually essential to you

corrideat

5+ Year Member



 
Msg#: 3945327 posted 7:24 pm on Jul 8, 2009 (gmt 0)

Actually, I am on a Linux server, so I'm afraid that I wouldn't be able to use that product, notwithstanding the fact that I could give it a try on a VM to get some ideas.
Besides, I'm asking what BradleyT quoted, that is, I'm requesting whether you could suggest more features. Before you mantion it, visitor origin (geoip) will be in the final product, but I haven't included it on the list because I listed the information gotten in the client side. When you suggest features, however, you needn't be limited by this: they may be both in the server or in the client.

incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 3945327 posted 7:43 pm on Jul 8, 2009 (gmt 0)

Actually, I am on a Linux server, so I'm afraid that I wouldn't be able to use that product

It works with Linux now with their SaaS version.

You're obviously using javascript to collect all of the information and now they're hosting the stats engine, you just plug in the javascript on your pages and that's all you need to do.

toplisek

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 3945327 posted 7:09 am on Jul 9, 2009 (gmt 0)

What does it mean Quriks or Standard mode?

corrideat

5+ Year Member



 
Msg#: 3945327 posted 4:12 pm on Jul 9, 2009 (gmt 0)

In the old times, each browser had its own behavior (up to NS4 - IE5.5.)
Modern browsers are supposed to follow the web standard. However, the "standard behavior" would crash websites designed to be seen in old such old browsers. As a result, browser developers decided to create the "quirks mode", and they render websites exactly as NS4 or IE5.5 used to.
Read more about it at https://developer.mozilla.org/En/Mozilla_Quirks_Mode_Behavior and https://developer.mozilla.org/en/Mozilla%27s_Quirks_Mode

Receptional

WebmasterWorld Administrator receptional us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3945327 posted 9:15 am on Jul 10, 2009 (gmt 0)

This thread seems to now be asking questions that are off topic. Please ask any more technical questions in the appropriate forum.

Thanks.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved