Forum Moderators: DixonJones

Message Too Old, No Replies

Python log file analysis source?

I'd like to write a custom log file analyzer. Other languages OK too.

         

MisterMarkup

10:47 am on May 27, 2006 (gmt 0)

10+ Year Member



I'd like to create specialized web server log file analysis reports for one my websites. I have the sense that I might have an easier time writing a script that used some kind of open source log file analysis library than trying to get an existing program to do what I want. I would prefer one written in Python but could work with most languages.

What I'm trying to do at the moment is to measure the effect on my traffic of passing out hardcopy handbills offerring a free download at my site. The url on my handbill is www.example.com but the download itself is at www.example.com/widgets

If someone typed in the URL from my handbill, their hit to my homepage won't have a referring page. Their click on my widgets link will have a referring page of my homepage. I can get reports of either of these things with Analog, but it won't tell me who did both - analog doesn't track visits, just requests.

Possibly there is another analyzer that will track visits, but sometimes I have more complex needs. It would be nice to have a way to script custom analyzers for such purposes.

I believe I am able to see some effect on my traffic, but it's very small. I passed out about five hundred handbills at a festival. While most people took them enthusiastically, I get the impression that only about one percent visited my site. I don't yet have a way to really tell how many of them downloaded my file.

MisterMarkup

1:54 pm on May 27, 2006 (gmt 0)

10+ Year Member



It looks like I can get the tracking I want with AWStats [awstats.sourceforge.net], but it wants to be installed on the web server, and to work on one particular log file.

I'm on a virtual host, and have several domains. I download log files to my home computer to analyze, and keep each month's file in a different directory. The awstats_configure.pl script says that one should do a manual installation if one is not the server administrator, and refers me to the documentation, but the documentation doesn't really say how to do a manual configuration.

Perhaps you can give me some tips.

Jack_Hughes

6:05 pm on May 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



it would be easier if you put a unique url on your flyer. something like www.example.com/freeoffer or something like that. don't link to that url anywhere else except on the flyer. then, any hits on that url you know are a consequence of the flyers.

as for custom software. there are a bunch of options. python has this [linuxgazette.net ]

perl has [search.cpan.org ]

But, you may find that Google Analytics is your best bet. you can set up the download to be a site goal and then track, using a number of different factors, where your conversions come from. Oh, and its free! Could save you a whole heap of time. Only write code when you have to.