Welcome to WebmasterWorld Guest from 50.16.84.67

Forum Moderators: bakedjake

Better log analysis for Linux

Webalizer just doesn't cut it.

   
9:13 pm on Oct 2, 2002 (gmt 0)

WebmasterWorld Senior Member nick_w is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Hi all, looking through my webalizer stats I'm finally realizing how woefully inadequate they are.

Presuming I can download the raw log files to my Linux box: What's my best option?

I'm noticing inexcusable descrepencys between my raw files and my stats. My site gets a hit a few times an hour for one phrase and it does not show in my webalizer stats at all!

Apart from that, I just need more informaion ;)

Thanks

Nick

9:19 pm on Oct 2, 2002 (gmt 0)

WebmasterWorld Senior Member digitalghost is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Sawmill is pretty decent, but I prefer FastStats. Sawmill has the advantage of being free.
9:27 pm on Oct 2, 2002 (gmt 0)

WebmasterWorld Senior Member nick_w is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Right! Well it seems I have Sawmill so:

$> man sawmill

it is then ;)

Cheers...

Nick

9:27 pm on Oct 2, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



'analog' is the standard against which all others seem to compare themselves, so you might want to try that. It's extremely configurable and I am only beginning to think at all about website statistics, so I have little experience to base any analysis on, but there you go. There's also something called 'awstats' that claims to be better and easier to use than webalizer.
9:30 pm on Oct 2, 2002 (gmt 0)

WebmasterWorld Senior Member nick_w is a WebmasterWorld Top Contributor of All Time 10+ Year Member



....er

Sawmill is/was a windows manager (now called sawfish) no man entry and doesn't appear to be the right tool anyway...what gives?

Nick

9:30 pm on Oct 2, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I thought Sawmill was a window manager... in fact, I'm pretty sure it's the one I'm using :)
9:42 pm on Oct 2, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Nick_W - might you be able to set up webaliser a little better?

Webaliser needs you to tell it the preceeding code before the search term for each search engine - in its configuration - for it to correctly identify the key word from the search / referral string. If you dont check that for every engine then you may not get correct key terms results.

After all its not as if all the engines keep the same referrer information over time. Compare to your raw logs and adjust accordingly.

imho for free you dont get much better value for money than webaliser :-)

Example from a standard config ... (may not be up to date)

#MangleAgents 0

# The SearchEngine keywords allow specification of search engines and
# their query strings on the URL. These are used to locate and report
# what search strings are used to find your site. The first word is
# a substring to match in the referrer field that identifies the search
# engine, and the second is the URL variable used by that search engine
# to define it's search terms.

SearchEngineyahoo.comp=
SearchEnginealtavista.comq=
SearchEnginegoogle.comq=
SearchEngineeureka.comq=
SearchEnginelycos.comquery=
SearchEnginehotbot.comMT=
SearchEnginemsn.comMT=
SearchEngineinfoseek.comqt=
SearchEnginewebcrawlersearchText=
SearchEngineexcitesearch=
SearchEnginenetscape.comsearch=
SearchEnginemamma.comquery=
SearchEnginealltheweb.comquery=
SearchEnginenorthernlight.com qr=

Hope this helps

10:01 pm on Oct 2, 2002 (gmt 0)

WebmasterWorld Senior Member digitalghost is a WebmasterWorld Top Contributor of All Time 10+ Year Member



There is a Sawmill for Linux version folks. A search will find it. Some of my clients have access to raw logs, others do not so I have to use whatever I can to get the best stats I can get. Out of all of them, Webalizer seems to be the weakest.
10:21 pm on Oct 2, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi digitalghost I see you are pretty sold on sawmill I have never seen that one .. I think if Nick_W confirms that his webaliser is configured correctly he might like it rather better (again as I mentioned cost benefit does not get much better) I find it fine as a basic quick check on activity - there are always logs and offline dbases for detailed work.

fwiw I hated analog the one time I had to look at it.

The online one I like the best is urchin .. allows quick easy deep drill down into individual search engine referrals etc etc all online .. the only negative I could see was that the version I get to play with does not provide paper reports which some larger clients would rather have than online access ..

10:29 pm on Oct 2, 2002 (gmt 0)

WebmasterWorld Senior Member nick_w is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Hi,

Thanks everyone. Can't confirm if Webalizer is configed right as I'm just not up to huge edit jobs tonight ;)

But, I can confirm that Sawmill is absolutely great! just gotta check out what it'll cost me after my 30 days is up ;)

Nick

7:08 am on Oct 3, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



We use webalyzer on all our accounts.

It is excellent at what it does once you realise its limitation.

If you modify the config files wou can get a lot more out of it than the default settings - which frankly are poor.

However the big 'down' is that it won't show key phrases by engine - ie you get key phrases and you get search engines - but you can't see which came from where.

For this I use 'Relax' - its a free perl script which I run locally on downloaded files. It specialises in Search Engine referral analysis and is better than anything else I have tried - very detailed and covers loads of engines.

1:19 pm on Oct 3, 2002 (gmt 0)

10+ Year Member



I also had a look at Sawmil, and played through its samples. Very nice product, but too bad it is not free.

Personally I use AWStats [awstats.sourceforge.net] which is available at Source Forge. It is driven by Perl so it is a bit slower than Webalizer. However it provides a lot more information regarding to bots, search engine referrals, browser/OS identification, etc. Most importantly, it is in active development, whereas webalizer has not had a serious update for ages...

2:02 pm on Oct 3, 2002 (gmt 0)

WebmasterWorld Senior Member nick_w is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Digital ghost showed me this link:
[sawmill.net ]

Where you can get it free! ;)

Nick

8:44 pm on Oct 3, 2002 (gmt 0)

10+ Year Member



I've used webalizer, AWStats, http-analyze, Analog, and a few others.

The most correct statistics are produced by Analog but they are not as fancy as the others. Get Report Magic if you want fancy reports.

 

Featured Threads

Hot Threads This Week

Hot Threads This Month