Raw Logs & Referal URL's

Program to scan raw logs for url's?

[financialhost]

Does anyone know a good program ( preferably free ), that will allow me to scan the raw logs for referal URL's. Would be a bonus if it also contained prefilled URL's for search engine url's allowing me to see which keywords were used to reach the website from Google, Yahoo, etc.

The tools and way data is collected at work is causing a lot of problems and is not accurate, so this would be the idea way to know for certain how many referals we recieve from certain URL's.

The raw logs are very large - over 20GB and i would need an option to set the queries by date range.

Any help appreciated.

[Staffa]

analog.cx

[financialhost]

I am really looking for one that allows me to run sql queries so that i could check a particular URL aganst a date range.

For example i could query the following URL for a 4 day period,
[google.co.uk...]

And know how many times someone reached the site from Google UK using the search term widgets.

[cgrantski]

LogParser

[dcrombie]

You can list all referers from the logfile using the command-line:

awk -F\" '{print $4}' combined_log ¦ sort ¦ uniq -c ¦ sort

Should be easy enough to pipe that into an SQL database for analysing, but much easier to have that information in SQL to start with - using server-side scripting to record referer information and query strings.

[cgrantski]

"awk" assumes unix or linux and can't be done in Windows without Cygwin or the like.

[dragan djapic]

Awk and other command line tools for windows can be found if you search internet for "gnuwin32".

You don't need cygwin or anything else.