Forum Moderators: DixonJones

Message Too Old, No Replies

Pre-processsing web logs (scheduled)

How-to make my logs smaller.

         

CMwebmaster

8:45 pm on Jan 22, 2004 (gmt 0)

10+ Year Member



I need a way to pre-process my log files. I'm trying to strip out .js, .gif, and .jgp files from the logs BEFORE they are analyed. I have Webtrends to do the actual analysis, but I want to shrink the logs before Webtrends ever touches them. (I've found the documentation for webtrends script to be very sparse)

Also, I need to schedule this to run nightly, so the solution needs to be able to be run from the command line.

If anyone has a sample script, routine, program, etc that they know of, it would be greatly appreciated.

mbauser2

5:27 am on Jan 23, 2004 (gmt 0)

10+ Year Member



If you're on a UNIX system, "man grep" to read the instructions for grep(1). The "-v" switch is your friend.

If you're not on a UNIX system, live and learn.

jatar_k

5:50 am on Jan 23, 2004 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



Welcome to WebmasterWorld CMwebmaster,

try the sorting lines in here
Getting PHP session variables into your Apache logs [webmasterworld.com]

if you have access to the conf, otherwise mbauser2 probably has the best bet.

CMwebmaster

3:03 pm on Jan 23, 2004 (gmt 0)

10+ Year Member



mbauser2 wrote
============================
If you're not on a UNIX system, live and learn.
============================

I'm using IIS 5 on Windows 2000 (not by choice). I'm probably going to need a script that strips down the log files.

I'm living & learning....but, surely someone else has had this problem on IIS & Windows?

jatar_k wrote:
============================
Welcome to WebmasterWorld CMwebmaster,
try the sorting lines in here
Getting PHP session variables into your Apache logs
============================

Re welcome: Thanks! Great forums...glad I found them.
Re apache: I'm using IIS :( I'll have to strip down the logs after they've been written by IIS.