Managing access log files using webmin

Need to split and compress

[louponne]

I've been pretty-happily using webmin to manage a dedicated server for a few months now. I've learned to do all the basic tasks.

However, I just went to download the access logs of a specific site, and notice that each site has one - huge - access log file (on busy sites, the files is - gulp - over 1Go). On other servers, I'm used to seeing daily log files, and they're compressed. So obviously here is something I don't know how to do - I've looked on webmin and checked my handy-dandy "webmin book", but can't find how to set this up.

Can anyone help me on this one?

[louponne]

The more I think about this, the more I think that the question shouldn't have mentioned webmin.

I'm betting that this task is accomplished via a cron job? (though I still don't know how to do it!)

[StupidScript]

You should consider using "logrotate", which is installed on many Linux distros. Logrotate is configurable for how large each log gets to be before it's archived, and so on. It may already be in your "System->Scheduled Cron Jobs" section of Webmin, but not active. There should also be an entry in your System section for "Log File Rotation", which is where you can coonfigure an installation of logrotate.

[louponne]

Thanks for your help! :)

ok, I have found System / Log File Rotate

I see that I can set up a daily rotation, and have the old log files compressed. So far so good. If I set the "Number of old logs to keep" to 90, I guess that will limit the old log files the 3 months' worth. I assume the oldest ones are automatically deleted?

Is that pretty much it?

Do I have to add a line for each log file - two per site, for the access_log and the error_log - or can I run one commmande to rotate the 60 or so log files I now have accumulating?

[edited by: louponne at 11:08 pm (utc) on Feb. 21, 2005]

[StupidScript]

Each command is for one log file ... bummer! But on the bright side, you only have to set it up once. And you are correct, it will delete the oldest logs when it reaches your configuration setting. Note, too, that if you run out of disk space, logrotate will send the rotating log to /dev/null ... basically deleting it (the old one), so keep an eye on your resources.

Have fun!

[louponne]

hm, another question - I see on other servers that it's the access_log that's archived. I don't see the error_log archived at all - is this standard practice?

What's done with the error_log - is it rotated?

[StupidScript]

I don't rotate mine, but then, they don't get very big unless there's a problem. (Typically I'll keep a close eye on the error logs for the first couple of weeks of a site installation, fixing errors as they appear, until there are hardly any errors. After that, it's usually hack attempts that generate errors, and I filter those off into a separate "hacker.log" anyway.)

Here's my hacker log spec in httpd.conf:
(Note: 3 lines: LogFormat, SetEnvIf, and CustomLog)
====================================================

LogFormat "%a %{SID}e %l %u %t \"%r\" %>s %b \"%{Referer}i\"" hacklog

SetEnvIf Request_URI \.ida¦\.exe¦\.dll¦OPTIONS¦CONNECT¦\.cfm¦\/race\/¦formmail¦FORMMAIL¦Formmail¦MSOffice¦\/ctruls\/¦\"¦<¦>¦XXXXX dolog

CustomLog logs/hacker.log hacklog env=dolog

====================================================

Regardless, you _can_ set up logrotate to rotate your error logs. Just make entries for them, as the others.

[louponne]

Many thanks again for your help! :)

I don't rotate mine, but then, they don't get very big unless there's a problem. (Typically I'll keep a close eye on the error logs for the first couple of weeks of a site installation, fixing errors as they appear, until there are hardly any errors....
Regardless, you _can_ set up logrotate to rotate your error logs. Just make entries for them, as the others.

Yes, this definitely sounds like the way I'll do it!

[louponne]

ok, I have finally gotten this to work.

Once the logs rotate, the new access_log files are created, but then they sit there and don't log anything at all. The only way to trigger logging again that I have found is to reboot Apache.

Am I missing something here (again)?

[StupidScript]

Yes, you can:

apachectl restart

Not so bad, really.

Or you can send the syslog daemon a SIGHUP signal to re-initiallize it:

syslogd -SIGHUP

Check

syslogd man

and

services man

for more info.

[louponne]

Yes, you can:
apachectl restart
Not so bad, really.

Or you can send the syslog daemon a SIGHUP signal to re-initiallize it:
syslogd -SIGHUP

okay, so this is a necessary step then. Any preference between the two methods you suggest?

[StupidScript]

syslogd -SIGHUP

simply gets the log files going again, where

apachectl restart

does more, of course, but neither one has any effect on anything else (i.e. active sessions, etc.), so unless there's a problem with your httpd.conf that makes restarting Apache take a long time, either should do.

I've got an older server that includes the Apache restart at the end of the script and a newer one that reinits syslogd, and they're both working fine.

Might as well stick with

apachectl restart

, if that's what you already have in your cronjob script.

(I do stand to be corrected if someone has any observations that modify my statements, above. :)