Forum Moderators: phranque
Many of the regulars here can read these logs better then plain English, unfortunatly, although I can muddle through, I'm not one of them. As a result I prefer to use a log analyzer program. These programs will dig through your log files for you and create various reports that help you make sense of the information. The reports they provide can usually be customized to meet your needs as well.
Two of these programs that I have personal experience with are WebTrends and Fast Stats. I haven't looked at WebTrends for a long while because Fast Stats meets my needs, I like it, and it's about $99. If memory serves WebTrends costs several hundred dollars and is much slower. Here's the Fast Stats [mach5.com] web site and you will likely learn a bit more there.
So how we doing so far? Getting close to answering your question? :)
More guesswork, if WebTrends generates the error you describe, the format of the log file is different then the format Web Trends is expecting. I'd be surprised if Web Trends doesn't allow you to select from amoung several standard formats, I think it has a feature to attempt to the identify the format and analysize accordingly. Bottom line is, I think Web Trends doesn't recognize your log format either.
Here [mach5.com] is some information on log files from the Fast Stats site, about midway down the page is a section entitled "I Host the Web Site Myself" which has links to pages that discuss commonly used log file formats. It's the best I can do with what you've told me so far, hopefully someone else can jump in and help out.
Most log analysizer programs are designed to work with the standard files as described at the Fast Stats site and little else. If your host provides something different (many do) you're pretty much stuck with the reports they provide or getting a new host...
It is crucial that you adjust and configure the program right for e.g. filtering. You should not use the defaults.
What WebTrends product are you using? What version?
Is noone here using it? I've just started and it seems pretty damn good to me.
my hosting company offers reports produced using Analog and allthough it's free I do find it quite hard to understand, they do have a link up with another company called report magic that will produce reports in graphical display specifically for use within analog, sorry don't know how much but here is the url www.reportmagic.com.
deltree
I have been for some time tring to get my head around understanding analog and failing hopelessly, could you give me some tips on understanding the report it produces, as I've hunted high and low on there help page but most of the information there relates to setting up analog not the reading of it.
If you don't want to bore people as it seens most don't use this service you could always e-mail me direct
thanks
Suffice to say that I've been using Analog for precisely 2 days so I am as much at sea as you are.
I am a Graphic Designer of the old school (pre computer) and pre-press professional so this web stuff is all pretty new to me.
My company manages a couple of sites on behalf of clients where we were using cgi counters (Yes I know they are horribly naff - now!) and my aim was to move on from there - hence Analog. We are trying it for two reasons. (a) It's Free. (b) The ISP that hosts these sites (Demon) generates seemingly comprehensive log files and recommended Analog.
I am open to any advice or tips from the pros here!
You can always get me on:
del@sanspam.freeuk.com
I come from a background in brick and mortar retail. I'm used to lots of hard data and straightforward Q&A about the business. But on the web, the more precise my questions about traffic became, the more difficulty I had being sure that I had an answer to the question I thought I was asking -- and I haven't got the skills needed to do a lot of tweaking to the software.
So I bought FastStats, and for a while thought I had it licked -- all the filtering that's built-in to their GUI is pretty groovy. And then the ceiling fell in on me -- this reported number versus that one didn't make any sense at all. So I wrote Fast Stats and they agreed that, yes, there were limitations, especially when it comes to proxy servers (like AOL).
This area of server logs analysis is a real Pandora's Box. Some of the problems are inherent in the nature of the web. Questions that seem natural (how many different people came to my site today?) are really rather complicated -- and a lot of that complication comes in order to preserve privacy for the visitor. When I look at it squarely, I know I couldn't answer a question like that about a brick and mortar store either, to any great degree of precision.
I think the most important part of log analysis is figuring out exactly what question you want to ask, and then asking exactly that question. It's not as easy at it would seem. Some questions are actually un-askable, not just un-answerable!
Let me suggest a couple things. Read the web pages and any help files you can find associated with Analog, Fast Stats, and Web Trends. Download and install the free trial version of Fast Stats from [mach5.com...]
It's good for thirty days and you should be able to get a pretty good idea of what info is in your log files and what all of these programs are about in that time.
As your understanding deepens, you'll likely be able to post specific questions and there are many here that will be happy to share what they know.
But we all need to be speaking the same language first! OK?
just on log analysis, before we used analog we just dowloaded the file into Excel and with a few tweaks were able to anwer any questions we wanted, by sorying, filtering etc.
All log files are is a delimited database anyway, with each hit as one record.
Nothing magic there, and all Webtrends, FastStats (which we have tried) and Analog do is to pretty it up and add a few bells and whistles.
There also is an ANalog mail list discussion group (there is a link somwhere on the Analog site) which we find useful.
But the first rule with any system is filter out all the gifs, jpgs, js, css and other files to leave yourself with just the hits that matter. Thats really all you need if your purpose is it know what pages are being hit, where people are coming from, kwywords used etc. (marketing intelligence info) Most people just need this unless you are an ISP calculating bandwidth etc.
Once you are just checking actual pages, it gets much clearer.
The other thing to remember is that bells and whistles beyong a point creates data that can be misleading. There is a limit to how much you can wring out of server logs!
The cache problem is one that any program will have problems with - not just Fast Stats.
To be ungracious, the mass of pretty data that Webtrends (for example) outputs, makes the result look far more authoritative than the raw data set actually provides. And gives people over-confidence in interpreting results.
Second rule, its best to know how to interpret what each line means in a raw data file before taking any hits analysis program for granted. It really is not hard to understand at all. But of course the number crunching is what you paying for.
