Well, here's a scenario I came across today:
- a cron job appends to a file, using exclusive lock
- a cgi uses this file to build a page, read only, so no flock
- when two instances of the cgi are running at the same time as the cron job, all three processes hang, system load rockets, and often restarting apache is the only way out.
I switched to using exclusive locks on every access, and the problem seems to be solved. I can't see a logical explanation for this - any ideas?
(Using Perl 5.004 on RH6.2)
>appends to a file, using exclusive lock
You shouldn't have to flock an append - I don't. Take the active list here. It gets abused with reads and writes all day long. The only time I flock it is when a read-modify-write sequence occurrs (replies), but on new posts (append), I never do. Nor do I use it when replies are written to threads. (I'm about to add this reply - no flock on append - no problem).
I can't see how the cron fits into it any different than a normal server request fulfillment.
I don't know if any of that helps in your situation.
Hmm... the file in question is used by several cgis, including some that read-modify-write. Suppose the file is appended to during the modify stage of another script? It's your classic 'race condition' - Bam! the appended data is lost... or am I missing something?
"Such discretionary locks are more flexible, but offer
fewer guarantees. This means that files locked with C<flock> may be
modified by programs that do not also use C<flock>."
That reads to me that the append is allowed to happen if no flock is set for it?
If one process accessing a file uses flock, then all other processes accessing the same file in whatever way have to use it as well, or interesting things will happen. The flock calls don't block anything, they only create "advisory locks". Which is quite similar a concept as "cooperative multitasking"... ;)
Yep, just tested this with a couple of scripts. The appending process ignored the read-modify-write lock.
Depending on the speed of the modification script, the window of opportunity for corruption is miniscule. But a race condition will bite eventually, you can bet on it.
Lucky indeed!
On the other hand, if you're talking about the forums here, the editing operations are probably very few and far between, compared to the appends. If I'm calculating correctly, then your figure sums up to an average of 20 messages added per hour, which in reality probably oscillates between 1 and 500. This means that, in an estimated worst case, you get less than ten appends per minute, distributed among more than 30 forums and probably hundreds of threads. slipping a few edits in between there only takes a lot of luck, not a hell of a lot... ;)
I just didn't realize that. BOTH the read and write processes have to use flock. I thought it was at the file system/os level.
