How reliable are Apache server logs ?

forum admin trying to catch writer of anonymous mails

[Berliner]

I'm an admin in a forum and the web server software is Apache. I'm new to websites in general as well, so I could use some help.

One of our forum members was receiving anonymous e-mails, which due to the use of remailers, had been impossible to trace.

Recently, though, one of the e-mails contained a link to a post in our forum, which had only been "live" for 10 minutes before the e-mail was received. Our forum is small, so I downloaded the server logs to see who might come into question.

Unfortunately it seems that the only people who read the post in the 10 minute time span are myself and the author of the post, a person who is well thought of. On the one hand I have no tolerance for writers of anonymous e-mails, on the other hand I don't want to accuse someone falsely, hence the question(s):

Are Apache server logs 100% reliable in determining who has accessed the website ?

Is it possible in my case, that someone had read the post within the 10 minute time span, but it was not recorded in the log ?

I would appreciate any insight.

Thanks,
Berliner

[Samizdata]

Are Apache server logs 100% reliable in determining who has accessed the website

Logs are zero per cent reliable in determining "who".

Logs can - if so configured - be broadly reliable in determing "what".

But I wouldn't say they were 100 per cent reliable in anything as glitches have been known.

And they can be edited by anyone who has access to them.

...

[Berliner]

Thank you very much, Samizdata.

But I wouldn't say they were 100 per cent reliable in anything as glitches have been known.

Is it possible to see a glitch in the server log, or could it just be that a few entries are missing with no other indication ?

[topr8]

be careful, i assume you're in the EU

you could be in breach of all kinds of privacy laws if you start trying to personally identify people as having done something.

i'm not a lawyer but i know the area is complex

[wilderness]

be careful

I agree!
Given the vulnerability of many computers to malware and actions by software's beyond a users knowledge, it'd be quite easy to falsey accuse somebody, whom never had that intent.

Additionally, eMail headers (especially from the free-mail-providers) are not as detailed as they once were.

[Berliner]

Thank you very much. If there is any chance I could be wrong, I would rather not do anything.

Could someone answer my question here please ?

But I wouldn't say they were 100 per cent reliable in anything as glitches have been known.

Is it possible to see a glitch in the server log, or could it just be that a few entries are missing with no other indication ?

Thanks,
Berliner

[wilderness]

"Generally speaking" raw logs are reliable.

That doesn't prevent failed writes or script errors when logs are generated, which happens for either complete entries or at times, even partial entries.

The "assumed" interpretation (or misinterpretation) of logs may be the biggest issue.

It's possible to use other tools (email headers, and even an outside website (and refers of same website)) to "point towards" the confirmation of an identity, however these tasks are dependent upon both your experience and depth of activity.

A final confirmation is certainly not absolute.

As I stated previously, many users today are infected with malware and offer no control of their machines activity.

[Berliner]

OK, wilderness, thanks for the reply.

It sounds as if a server log can be an effective tool, but may not be sufficient as evidence on its own.

I do appreciate all your answers, and will keep them in mind.

Berliner

[jdMorgan]

There is the possibility that the server could have "dropped" a log entry.

There is also the possibility that if the forum thread URLs are created in a predictable manner, the link could have been written before the thread existed. This would of course depend on whether the thread content itself was cited in the e-mail, or if only the link was presented -- you will have to interpret this possibility.

You could always block the thread author's IP address and see what happens when you tell him when he complains that he "apparently" got blocked because you blocked the IP address of the sender of an unwanted e-mail...

Jim

[Berliner]

There is the possibility that the server could have "dropped" a log entry.

Would this be visible in an error log or otherwise ?

Thanks,
Berliner

[Pfui]

Permit me to take a slightly different, definitely too-long tack...

1.) I err on the obsessive side when it comes to log-reading and in 14-plus years of running two Perl-based boards on an Apache server, every single post-related oddity on the boards appeared in one or more logs (Combined Log Format; access, error, & rewrite). The oddities were wholly related to the Perl scripts' functions under rare circumstances and were 100% unrelated to who-what accessed which post when-where.

2.) Thus to me, the likelihood of your Apache server not correctly 'seeing'/recording/logging all hits to a single, 10-minute-old post is extremely slim. Slightly more likely? The author of that post has fellow posters' addresses in their mailreader, their machine is compromised and a botnet is blasting where they go to who they know. But even that strikes me as highly unlikely given the example time-frame, but primarily because inbox infections spam their own stuff.

3.) Momentarily slipping into non-Apache territory... Hate to say it but I'd be more likely to suspect the individual(s) than your server or their machines. Over the years, I've found a few of the most reputable, well-comported, apparently sane posters to be startlingly iffy, erratic, and/or delusional in e-mail as evidenced by the thankfully rare, out-of-the-blue claims that 'another poster hacked my machine' or 'so-and-so is reading my e-mail.' Etc.

So what do you do?

4.) Were I in your shoes, and particularly since you're new to websites (and forums? and logs?) and given the privacy issues into which you could inadvertently find yourself quite mired, and given the iffiness of people, machines, and programs, I'd extricate myself ASAP and:

-- Thank the anon e-mail reporter for their info and mention they may find it helpful and common to use a free account, @gmail, for example, when they post anywhere.

Of course, if anonymous e-mails contain personal threats, the recipient should contact their local police. However if the messages are spam/phishing, the recipient (not you) can anonymously report them via SpamCop.net. Copy-paste headers+message and Spam Cop.net traces the tracks (very interesting stuff) and optionally reports the spam to the responsible ISP(s).

-- Resist saying anything to the post author about any of this because there are no facts implicating their hardware/software. Resist saying anything to the anon e-mail recipient for the same reasons.

-- Make sure your forum's scripts are always 100% current in terms of security, particularly all membership/login files. If your scripts are PHP-based, security will require near-constant vigilance because PHP scripts are ceaselessly probed and successfully exploited.

-- If you have posters' e-mail addresses on your personal machine, say because you get e-mail notices to pre-approve posts, make sure your machine is 100% clean, too. There are few common denominators in this scenario and you're one of them:)

Okay. Finis. Finally!

[SteveWh]

If any search engines crawled the post during that 10 minutes, someone could have seen it in the search engine cache, which would not be recorded in your access logs, except maybe for images and other secondary files. Search engines sometimes index new forum posts very fast.

I agree that the email's reference to a particular forum post isn't sufficiently reliable evidence of anything useful, even though Apache access logs are generally very reliable. There are potentially confounding factors besides just the issue of whether the log entries are complete and accurate.