Web Log Statistics

How often do you check?

[madcat]

I'm wading through the Analog help files, one question off the top of my head would be, how often do you run your log files? How do you actually get weekly, monthly or yearly statistics? Would I have to download the log files at the same time each day and then run them all at the same time, say at the end of the week, month or year? Or do log files keep a running track of what's happening for a certain amount of time anyway? Could I download them periodically or just at random times and make assumptions based on the ones I download. It's all very confusing. What are some ways to think about managing site logs?

Thanks very much for any help

[Macguru]

Hi madcat,

I usually do traffic reports once a month.

Sometimes spot checks are good for results on conventional media placement. Some clients want to see results of radio, TV or print adds before investing more in it.

>>Or do log files keep a running track of what's happening for a certain amount of time anyway?

This depends on your host. Some delete logfiles after a given time or size. The best service I get from remote hosts is they 'zip' logfiles montly for me to download and keep it available for a month after.

I can merge them later for yearly stats.

I usually ask analog to output COMPUTER format for later use.

[BlobFisk]

Hi madcat,

On one of my clients sites I do a quick log check every day (the site gets about 100,000 unique visitors per day). We are running Webtrends Live via the Reporting service and I find it invaluable for this.

I do however, one a month go through the logs in much finer detail, often passing them through Analog for a second opinion!

HTH

[HuhuFruFru]

only once a month macguru? i check my logfiles 2-3 times a day...

[Macguru]

Yep, only once a month HuhuFruFru. I dont have a single client who would bother with 2 or 3 traffic reports a day. I am curious about the benefits of checking logfiles that often.

Typical logfile size vary a lot for each sites. A lot of them are over a gig per month and take hours to download and 'crunch'. BlobFisk care for a site with 100.000 uniques a day. I believe he is not storing the files on a floppy. ;)

[madcat]

Thanks for sharing some of your thoughts on the matter...

My host deletes log files every Sunday evening. So to get a month of stats, I would need to download the logs once a week before they're about to remove them for a month, and then run them together. That sound right?

Getting a .zip of the entire month sounds streamlined, that would be nice.

[BlobFisk]

I believe he is not storing the files on a floppy.

It would make things a bit easier if I could! :)

In fairness, I only have 2 clients where logs require this level of effort. All the others would typically be once a month (unless there is a campaign that should be generating increased traffic), although I would take a quick peek once a week to see how things are going.

I really enjoy analyzing log files (most of the time)... it's interesting to see variations and taking meaning from the statistics.

[Macguru]

>>That sound right?

Yes that sounds right. Are they storing the files for a bit before they delete them?

Otherwise you might have 'holes' in the merged month.

[HuhuFruFru]

macguru, i check my logfiles 2-3 times a day to check what the robots (and the users of course) are doing, so that i can make changes quickly and without any delay. once a month is just unimaginable for me :)

[Ankheg]

There is something to be said for checking your logs daily (or more often), as I learned from personal experience.

One of my sites is about fifty pages, gets about 100 visitors a day, varying; about 2MB of HTTP traffic per day, including about 300kb per day from a banner exchange. Anyway, a small niche SE's spider had beaucoup problems with my forum (PHPBB) one fine day - didn't like the SID's. It hit my site (actually, an error page - PHPBB exhausted it's sessions table and went splat) just over 1,000 times per hour for about six hours straight, before I checked the logs and noticed. I called the fellow running the SE, and he was completely unaware of the problem. Lessons learned: Ban all robots from /forum, and check the logs frequently for signs of problems...

[Brett_Tabke]

I check mine about once an hour (or more) when I'm at the kb. Sometimes I keep a 'tail' open on certain logs to look for problems or problem visitors.

[dcheney]

I get my logs about once a day (helps keep the download size reasonable). About all I check on a daily basis is for errors and major new referal sources. If the volume is out of its normal range then I'll also look into that.