Welcome to WebmasterWorld Guest from 188.8.131.52
I'm afraid I'm frequently guilty of this. I have a dual-monitor setup at work, and one screen frequently has several windows showing the live logs of various webservers.
Interesting to watch, and I sometimes go as far as to filter it to show only web page requests, and then follow visitors progress live through the website. Useful to watch when you've added a special offer to the homepage, and just for gauging surfers' habits.
I'm "lucky" in that my commercial site has steady but not overwhelming traffic, such that real time log watching is possible - there is rarely more than one person on the site at a time.
I do a couple of things with cookies and Apache configuration to make this a more manageable process.
Firstly, I have a "secret" page on my site that sets a cookie called "nolog". I then have a bunch of SetEnvIf lines in httpd.conf that decide whether or not to log a request.
They first line is:
SetEnvIfNoCase Request_URI .* normalrequest=true
Which sets an environment variable "normalrequest" to start off with the "log everything" case.
SetEnvIfNoCase Request_URI \.gif$!normalrequest
SetEnvIfNoCase Cookie .*nolog.*!normalrequest
...which clear "normalrequest" if the request is for a .gif image or from me (because of the nolog cookie). Similar lines kill logging for .css, bots etc.
This is used together with...
CustomLog [existing parameters] env=normalrequest
Probably an easier way to do all that using .htaccess and subdirs. and what not, but it only took a few minutes to setup.
Since developed a impulsive glance towards my logging monitor whenever my peripheral vision detects a shift up the screen!
SetEnvIfNoCase Request_URI "\.(gif¦jpeg¦jpg¦png¦css¦js¦swf¦pdf)$" notapage
SetEnvIfNoCase Remote_Addr "127\.0\.0\.0" notapage
With "127.0.0.1" being the IP of my workstation.
(I've realised that this is the wrong way to do it, though - it would be much easier to just set what file types are pages, rather than all those that aren't.)
Then it just pipes the results to a script:
CustomLog ¦/path/to/docroot/write-log.sh watching env=!notapage
"write-referer-log.sh" just reads the line in, checks if it matches some stuff I'm not interested in, and writes a line to the log if not. It kind of sucks because the script has to be loaded and executed (which in turn opens the log file, writes a line, and closes it) for every hit. It's not like the server's falling over or anything, but it's sort of offensive to anyone who cares about efficiency.
It would be nice to have another monitor though! I've developed a sort of nervous twitch - if I don't flip to the window with logs open once every couple of minutes it feels like something's not quite right...