homepage Welcome to WebmasterWorld Guest from 54.227.11.45
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Hardware and OS Related Technologies / Website Technology Issues
Forum Library, Charter, Moderators: phranque

Website Technology Issues Forum

    
Raw logs vs. customized logs (via CGI)
advantages and disadvantages
alexjc




msg:671786
 1:46 pm on Nov 11, 2000 (gmt 0)


I've been hesitating about this for a while. It's fairly difficult to extract exact information from raw logs (like unique visitors per day, see other threads), but as a webmaster you need to setup absolutely nothing to get this working.

For customized logs, you obviously need to run CGI for each page, and write to your own log file. This way you can easily track independant users with cookies, and all their session information: pages visited, links followed...

What do the rest of you use? Are there any alternatives i haven't looked into??

Alex

 

oilman




msg:671787
 8:36 pm on Nov 11, 2000 (gmt 0)

hi alex - welcome to WebmasterWorld.

raw logs are definitly a chore to wade through but you don't necessarily need a custom cgi script to handle logging for you. There are many good programs that you can buy (some are free) that will filter and analyze the raw logs for you.

Check out this thread for some suggestions:
[webmasterworld.com...]

spiky




msg:671788
 11:02 am on Nov 12, 2000 (gmt 0)

Could some one cover some basics for me. what is a unique visitor ( and may be a few other definitions of the data these logger tools produce. )

Is there a basic tutorial out there ??

Thanks

oilman




msg:671789
 8:08 pm on Nov 12, 2000 (gmt 0)

here's some basics. These are the two most confused for each other.

unique visitor: can also be looked at as 'unique ip' and pretty much means individual visitors. This number strips out multiple visits from the same IP over the given reporting period. For example: even if some one visits that site 50 times in one day they will only be counted as 1 unique visitor for that day.

hits: this is the most commonly misrepresented term online. The true definition is that hits refers to the number of times the server is 'hit' for a file call. If you have a 6 images on a page and a surfer goes to that page the server will record 7 hits - 1 for the page and 6 for the images.

What else would you like to know?

spiky




msg:671790
 8:47 am on Nov 14, 2000 (gmt 0)

If the page has been "cached" and the surfer clicks on the "cache" hyperlink, will this show in any stats ?

mark_roach




msg:671791
 3:14 pm on Nov 15, 2000 (gmt 0)

I assume you are are talking about the google cache.
If the surfer only views that page then you will be none the wiser. If they follow any of the links from your cached page then then that will show up in your logs.

tedster




msg:671792
 6:38 pm on Nov 15, 2000 (gmt 0)

mark and spiky,

There is a way to see google cache hits in your server logs.

Google caches only the HTML, but not the images (saving disk space, I'm sure). The images for your page are retained as links to your server. When someone clicks on the "cached results" button in Google, their servers call for the images from your server.

For each of these hits on images, instead of the simple searchphrase, you see cache:www.mydomain.com/ searchphrase.

Also, watch out for this one: each image request made by the Google cached page can come from a different IP. So if there are 7 images on a page, it looks like 7 uniques for that one click on the Google cache button. It can also look like the search phrase was used 7 times when it was only searched on once.

mark_roach




msg:671793
 8:00 pm on Nov 15, 2000 (gmt 0)

Tedster - Of course you are right, thanks for putting me straight. You have also highlighted another reason why the number of uniques is such a worthless figure.

Brett_Tabke




msg:671794
 8:20 am on Nov 17, 2000 (gmt 0)

Spiky, no a cache hit will not show in your logs using standard tools.

I assume you are talking a browser cache or proxy cache like AOL's. Those are not tracked by standard logging utils since the page is actually never requested from the server.

In order to track such page views, you would need to use a system like Tedster describe. One possiblity is using an "IMG" call tag to your counter that is CGI in nature.

Suppose your cgi logger is at "domain.com/cgi-bin/logger.cgi". In order to over ride a browser cache or a proxy cache you could do a couple of different things:

1- call your logger with a dynamic url "img src='domain.com/cgi-bin/logger.cgi?nothing'.

That type of call would over ride most browser caches and proxy cache. They won't cache a cgi generated file. One step better is to replace "nothing" with a random string so that the browser thinks that it is truely dynamic.

2- Using a http header on the IMG that restrict caching or uses the header EXPIRES tag.

Aol will honor the expires tag and most browsers will too (not all).

Those are the only real choice available. You can try a JS script with a dynamic doc write (again, a unique string), but your results will only be marginally better than nothing.

This is the hardest thing there is to tracking on the web - getting semi accurate data. The biggest hurdle is AOL's proxy cache. It can really throw your numbers for a loop. Especially if you have good rankings on AOL's Netfind search engine.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Hardware and OS Related Technologies / Website Technology Issues
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved