Updating a flat text file without rewriting

text file database trouble

[jt007superman]

Does anyone know how I could update a record in a text file without rewriting the entire file. I think this is the way a SQL server connection works, but I'm not sure. I have had problems with a large Perl/text database program I've written. When a user interrupts a connection to the text file, by say clicking a form twice, the file loses information because the write process is stopped midway, and a new process starts but only reading the file up to the stopped location. Any advice is greatly appreciated.

[txbakers]

I think this is the way a SQL server connection works

You can't compare a real database process with a text file.

A database has sophisticated row handling in place to update the one row you need updated. A text file, by nature, has to be read top to bottom to find the line to be updated.

You have some flexibility with text files, like a serach perhaps, but it still needs to be read top to bottom.

A database also has row and table locking to prevent the issues you describe.

[PCInk]

It can be done, but you'll need to access the files differently. Each open file has a pointer (of how far into the file you are). You can read and write at the place of the pointer or move the pointer about. One difficulty with text files is that line length varies, so unless every line in your text file is the same length then you will need to shuffle the lines under the one you are writing to compensate. Changing the length of the first line would normally mean rewriting the whole file anyway.

Another method is to 'lock' the open file so that the second task cannot open the file (ruining the first file process). Some OS allow file locking easily, others are more difficult. One simple method is to create another file and store one '0' in it. Every program that updates the file (or writes to it in any way) should update that 0 to a 1 (which takes a fraction of a second). Then when finished, reset the 1 back to a 0. Only allow a program to open a file when the content of the second file is a zero - I would choose to repeatedly attempt to open the file every few seconds for a specified time limit. After that it's up to you if you want to stop the program with an error or go ahead and write to the file (bearing in mind another program may be still also writing or another program may have crashed, losing the reset of the 0/1 flag).

Using an OS lock can cause problems in that if you do not unlock the file, it may cause lack of access to that file for all other programs until you manually unlock it again (this may happen on a server crash/program crash etc...).

[jt007superman]

That's about what I thought. The worst problem I've had is not a result of multiple users, but were the server process just quits prematurely. For example, if I were having a multi-user problem, I would be losing a record or two at once were two or more users had read the database file all at once before each had written to the file with their changes, while with my present problem the script will read 5000 records, and the process will quit after writing only 1000 or so. I should probably just go ahead and rewrite to a SQL so it's not possible to lose this much data at once.

Thanks for all your suggestions.