Welcome to WebmasterWorld Guest from 50.19.53.104

Forum Moderators: coopster & jatar k & phranque

Message Too Old, No Replies

flock(2) or not? Please comment on this project I'm working on

Share your opinions & experience if you think this is a good idea

     

explorador

6:03 pm on Mar 15, 2010 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Hi webmasters. I already know how to use flock() but I will deeply appreciate your comments and experiences on how much trust you put on this function before implementing a project. I've created online apps in Perl using <flat-files-databases> with no problem. Multiuser apps need file locking to avoid two people (or more) writing to the same file at once causing data loss or data corruption. Yes, I know databases do this but I'm still going to use Perl + flat files.

My way of implementing it: Whenever I'm using a file I first lock it, then read it-write to it and then unlock it. I made tests with flock(2) and if a user-process tries to read or write to the already locked file it will have to wait for the process to close-unlock the file before it can read it or write to it. This happens automatically without the need for the user to see any message and try again, the browser just waits milliseconds and the data is sent to it without any message or interruption, is transparent to the end user.

I had data loss and data corruption before using flock, not anymore so by now I'm happy with it. Then why do I ask? I need your comments:
  • Please share your general comments and experiences using flock on important data projects.
  • How much do you trust flock(2) ?
  • Would you use flock in the kind of project I'm working on?
  • Have you experienced data loss even using flock() ? **


* Using flock to lock files involves using it every time you access the file in every routine. In my experience and tests, also for read and write. Please correct me if I'm wrong.

The situation: a table of widgets (flat file database of 2,000 aprox lines with 30 fields) that will be read and updated many times a day by many online users. The thing is, the kind of data has me kinda worried so this is why I ask. For my peace of mind, I've solved this before using one flat file for each record (2,000 files) so, the chances of accessing the same file at once are really less than putting all the data on the same file but I don't want to grow the project to so many data files.

I could do it again if there's no other way (many files) but using a single file would be more appropriate and convenient for this project (I don't know how less secure it would be in terms of flock). That's why I will need your comments, the kind of data is related to money.

Thanks in advance.

janharders

6:19 pm on Mar 15, 2010 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



I didn't experience any serious problems with flock (that is: problems that couldn't have been avoided by thinking correctly), but I don't see the point in storing mission critial data in a flat file. A database would be much better, including row level locks, easy handling, indexes etc pp. Why avoid that just because you can?

Apart from that, I recommend taking a look at File Locking Tricks and Traps [perl.plover.com] that will get you some ideas for potential problems and their solutions.

explorador

6:54 pm on Mar 15, 2010 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Thanks, very interested info, already bookmarked. I agree with you on the DB, but the thing is this app will have constant backups online from server to server (perhaps hourly, every account) for transparent switching from one to the other in case of failure. Tricky, but I came up with an idea to solve it.

Thanks again.

Software error:

Can't locate /home/deploy/webmasterworld/code_format-v6.lib in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.18.2 /usr/local/share/perl/5.18.2 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.18 /usr/share/perl/5.18 /usr/local/lib/site_perl .) at decode-post-v6.lib line 27, <THREADDAT> line 5.

For help, please send mail to the webmaster (it@imninjas.com), giving this error message and the time and date of the error.