server logs

apache or OS

[smallcompany]

I had some question in the past where I got puzzled between Apache and i.e. Linux.

Now I have another question, but before I ask, here is the general one, just so I know this once forever:

When about server logs, is that Apache or OS (like some Linux form) matter?

Thanks

[mack]

The server logs are created by Apache, but the OS does create logs for various parts of the system.

The log for web traffic (Access log) is created by the server software, in your case Apache.

Mack.

[smallcompany]

Thanks.

In today's world when you don't need to know basics in order to run a site, it's easy to get confused.

Recently I spent quite a bit of time to figure how to get web access logs separated on daily bases.
It was Linux that I ended up playing with.

Now I'm back to Apache.

How to grasp the two since they don't refer to each other?

[mack]

If you have a Linux server you might want to think about installing a stats package such as Awstats It gives you a daily graphical display. It can be quite useful for analyzing your logs.

Mack.

[smallcompany]

Thanks.

The background of this question was the idea of doing conditional logging. I.e. organic traffic goes into one log, and PPC into other.

So far, with the help from other members, I learned that we could use "grep" to manipulate existing logs and export needed data into other text files. This would leave original access log intact.

On Apache side, I learned that there is SetEnvIf Request_URI which can be used for Conditional Logging.

This is a good start and I thank to both members for getting onto this (in Linux and Apache threads).

[jdMorgan]

Apache is the server application, Linux (or, for another example, Windows) is the operating system.

Apache creates the logs using the operating system's filesystem management functions.

You can write (and probably did write) a script to sort, archive, or otherwise manipulate your logs files. This script would run under the operating system, side-by-side with Apache, and it too would use the operating system's file management functions to create, open, read, and write files.

Apache, at its core, is only a URL-to-filename translator. It takes an incoming URL as requested by an HTTP client (e.g. browser or search robot), translates that to a filepath, and then asks the operating system to open and read that file. It then takes the contents of that file, adds a few HTTP response headers, and sends the result back to the client. Without all of the add-on modules that can be installed with Apache, it's really a fairly "tiny" program by today's standards...

Jim

[smallcompany]

Thanks.

That's the core of what I'm missing in (my) web story. I need to digest it more.

I got the Apache suggestion from "phranque" in Linux thread and I wonder what do you think about this:

[httpd.apache.org...]

and

[httpd.apache.org...]

My way of thinking tells me that any additional manipulation is better if it happens later, rather then real time.

My understanding is that this Apache solution would be real time, while things like using grep is whenever you want it, once the log file has been written.

I wonder what do you think about this, and how big a burden on the server would be with conditional logging on the Apache side?

Thanks

[jdMorgan]

Consider that many shared-hosting virtual servers are configured with conditional logging. Others are set up to "sort out" the log files for the virtual hosts on an hourly or daily basis. It's a trade-off of server load versus the usefulness and accessibility of the log files.

As a customer, for example, I will tolerate a one-hour delay on access logs, but I insist on real-time error logs, and I sometimes pay a premium to get all logs in real-time. So it depends on your "customer" and how important fast and efficient debugging is going to be to them.

Also, since "stats" are based on the log files, any delay in log availability is reflected as a delay in updated stats availability.

How big a burden... IMO, not much additional burden compared to logging for multiple virtual servers in the first place.

I hesitate to make recommendations simply because everyone has a different opinion, and I almost never see identically-configured servers -- though admittedly, I don't work for a hosting company where of course all (or most) servers would be configured the same way.

Jim