Web log analysis using data mining

[lwebzem]

Is there tool that can apply data mining or business intelligence algorithms to web log? Such algorithms could be clustering, classification, path analysis and so on.

How usable this can be in increasing web traffic? Does someone using this or it's something that is not usable unless you have huge website.
Are there any other forums or site that discuss or have resources for this technology applied to web log?

My host provided web reporting tool but it was just number of hits per referal, country, and so on. But I am looking something more than this.

Thanks in advance for your help.

[cgrantski]

Great question.

Path analysis, of a simple kind, is done by most of the more sophisticated analytics programs. It sounds like you have a really bottom-level program. Before jumping to data mining you should check the high-level ones. See the #1 item in this forum for a link to a list with descriptions.

I have never seen anything that does true data mining that individuals could possibly afford. For cluster analysis and other techniques in the predictive modeling area, I would mention Clementine from SPSS. It will process log type analysis with the addition of certain modules.

Yes, these are tremendously beneficial to have, because for sites with huge revenues a fraction of a percent difference in conversion will pay for the tool.

For the rest of us ... I have used Excel to process the output from WebTrends using cluster analysis and predictive modeling and so forth. It can be done. It can also be done with raw logs and Excel as long as the logs aren't too big. I guess Excel 2007 can handle far bigger files, so that would be where I'd start.

But it's a lotta work and, in my limited experience, usually only ends up confirming and precisely quantifying what a really smart and wise person would already know in a general way.

The shortcut to the same end result might be just experimenting with your site and using statistics from the analytics world to gradually improve the stats you want improved.

Sorry to sound cynical about today's web analytics programs, but when you step back, they are really just fifth-grade-level tabulations and arithmetic, done on a massive scale the way only computers can do.

[lwebzem]

Cgrantski,
Thanks very much for your response. It was very informative and i will need to think about this. And also will look more in high level programs.

Thanks and Best regards.