Flocking Advise

[\Dandy\ Randy]

Hey ... could use some advise on the best file locking method ... at present
I use the following:

#!/usr/bin/perl

use strict;
use 5.004;
use Fcntl qwDEFAULT :flock);

open (FH, "<data.txt") or die "Can't open file: $!";
flock (FH, LOCK_EX) or die "Can't lock file: $!";
$data=<FH>;
chomp ($data);
($total,$opened,$followed)=split(/\|/,$data);
close(FH);

$opened = $opened + 1;

sysopen(FH, "data.txt", O_WRONLY | O_CREAT) or die "can't open filename:
$!";
flock (FH, LOCK_EX) or die "can't lock filename: $!";
truncate (FH, 0) or die "can't truncate filename: $!";
print FH "$total|$opened|$followed\n";
close FH;

print "Content-type: text/html \n\n";
print "Done\n";
exit;

It seems to work ok, but am not sure if it is a good approach or not. One of
my main questions is ... what about sememorph locks ... been reading in
different places that you start you file open code by opening a dummy
sememorph file ... kinda like this:

open S, "> somefile.sem" or die ...;
flock S, LOCK_EX or die ...;
open F, "> somefile" or die ...;
# Now write F
close F;
close S;

What I see going on here is the code opens a file that does not contain
critical data, thus protecting the real data, and the user cannot open the
real file until the sem lock becomes free ... Two questions ... is this a
good approach, and secondly ... whats inside the sem file? is it an empty
dummy file? does it have to have a .sem extension? Thank you for you
opinions.

Randy

 

[David Efflandt]

Hey ... could use some advise on the best file locking method ... at present
I use the following:

#!/usr/bin/perl

use strict;
use 5.004;
use Fcntl qwDEFAULT :flock);

open (FH, "<data.txt") or die "Can't open file: $!";
flock (FH, LOCK_EX) or die "Can't lock file: $!";
$data=<FH>;
chomp ($data);
($total,$opened,$followed)=split(/\|/,$data);
close(FH);

$opened = $opened + 1;

sysopen(FH, "data.txt", O_WRONLY | O_CREAT) or die "can't open filename:
$!";
flock (FH, LOCK_EX) or die "can't lock filename: $!";
truncate (FH, 0) or die "can't truncate filename: $!";
print FH "$total|$opened|$followed\n";
close FH;

print "Content-type: text/html \n\n";
print "Done\n";
exit;

It seems to work ok, but am not sure if it is a good approach or not. One of
my main questions is ... what about sememorph locks ... been reading in
different places that you start you file open code by opening a dummy
sememorph file ... kinda like this:

One problem with what you are doing (hit counter?) is, what happens if
multiple instances of the script run at the same time? One script reads
it, closes it, another script opens it and reads it, first script opens
and writes to it, and second script modifies stale data and saves it.

My approach is to open it read/write ( +< ), flock it, seek the beginning
(just in case something else opened it before flock), modify and save
data, and then close it. Then another instance would have to wait until
flock cleared before it was able to flock, read and modify fresh data.
Otherwise your count could easily end up short.

 

[news]

My approach is to open it read/write ( +< ), flock it, seek the beginning
(just in case something else opened it before flock), modify and save
data, and then close it.

There's a very similar issue going on in a parallel thread at the moment.

What I really don't see is why there's a requirement to seek back to the
beginning of a file after it's been flocked. If you've not read anything
then the file pointer's at the start of the file anyway. (flock applies a
file lock rather than a partial region lock.) I appreciate you need a seek
when swapping between reading and writing, but that's not the issue here.

Can anyone give a concrete explanation as to why the seek is required
immediately after the flock?

Cheers,
Chris

 

[David Efflandt]

There's a very similar issue going on in a parallel thread at the moment.

What I really don't see is why there's a requirement to seek back to the
beginning of a file after it's been flocked. If you've not read anything
then the file pointer's at the start of the file anyway. (flock applies a
file lock rather than a partial region lock.) I appreciate you need a seek
when swapping between reading and writing, but that's not the issue here.

Can anyone give a concrete explanation as to why the seek is required
immediately after the flock?

perldoc -f flock

Basically something else could open the file (possibly for reading)
between the time you open it, and flock it, so the file pointer might not
be at the beginning of the file when you go to read it. It may not always
be necessary, but it is playing it safe, just in case.

 

[ctcgag]

Hey ... could use some advise on the best file locking method ... at
present I use the following:

#!/usr/bin/perl

use strict;
use 5.004;
use Fcntl qwDEFAULT :flock);

open (FH, "<data.txt") or die "Can't open file: $!";
flock (FH, LOCK_EX) or die "Can't lock file: $!";
$data=<FH>;
chomp ($data);
($total,$opened,$followed)=split(/\|/,$data);
close(FH);

Bang, you're dead! There was no point taking out an exclusive lock,
since you just lost it before the write occured.

$opened = $opened + 1;

sysopen(FH, "data.txt", O_WRONLY | O_CREAT) or die "can't open filename:
$!";
flock (FH, LOCK_EX) or die "can't lock filename: $!";
truncate (FH, 0) or die "can't truncate filename: $!";
print FH "$total|$opened|$followed\n";
close FH;

print "Content-type: text/html \n\n";
print "Done\n";
exit;

It seems to work ok, but am not sure if it is a good approach or not.

It is not. Both operations need to take place under the *same lock*,
not under two different locks. If it seems to work, you are probably
not testing under sufficiently contentious conditions.

One
of my main questions is ... what about sememorph locks ... been reading
in different places that you start you file open code by opening a dummy
sememorph file ... kinda like this:

You do not need to do this if flock works properly on your system.

One things you could do is open the file for read and write,
obtain the EX lock, read, truncate, write, close.

Another way is to use the file as it's own semaphore:
open FH for reading, obtain EX lock, read. *Do not close FH*
open FH2 for writing to the same file. (There is no need to lock
it as FH holds the lock on FH2's behalf). Write to FH2, close
FH2, close FH.

This last is the way I usually do it, as I've never bothered to
learn how to use truncate and +<.


Xho

 

[news]

Can anyone give a concrete explanation as to why the seek is required
immediately after the flock?

perldoc -f flock

The seek to EOF is correct when you're appending to a file. In the
general case (and particularly when reading) it's not required.

Basically something else could open the file (possibly for reading)
between the time you open it, and flock it, so the file pointer might not
be at the beginning of the file when you go to read it.

The file pointer is on a per-filehandle basis, not on a per-file
basis. We'd get mighty upset if I changed my file pointer and yours
updated too.

The notable exception is when *writing*, where we've specified that all
writing is to occur at the current EOF (i.e. append), in which case it's
correct behaviour).

Chris

 

[Alan J. Flavell]

One things you could do is open the file for read and write,
obtain the EX lock, read, truncate, write, close.

This is an FAQ, isn't it? It always makes me nervous to see
independent proposals offered to the readership when there's an
existing FAQ that addresses their requirement. If the FAQ is wrong or
incomplete or otherwise unsatisfactory, it seems to me that discussion
should focus on a proposal for a revised FAQ, rather than offering the
readership some kind of parallel-universe advice. No offence meant.

Another way is to use the file as it's own semaphore:
open FH for reading, obtain EX lock, read. *Do not close FH*
open FH2 for writing to the same file. (There is no need to lock
it as FH holds the lock on FH2's behalf). Write to FH2, close
FH2, close FH.

But now we're going in circles, because in a very recent posting on
this topic a contributor pointed out - and the FAQ says so too - that
on some platforms you can't get an exclusive lock without opening for
write.

See also the file-locking part of perlopentut (again, if there's
something wrong, the discussion should IMHO be contributing to fix it,
not just talking across it...)

This last is the way I usually do it, as I've never bothered to
learn how to use truncate and +<.

Well, I can only say that according to the available documentation,
your method appears to be less portable. OK, OK, as the FAQ says:

| Slavish adherence to portability concerns shouldn't get in the way
| of your getting your job done.

but where there's a choice, I'd try to incline to the portable one.

best regards

http://www.perldoc.com/perl5.8.0/pod/perlfaq5.html#How-can-I-lock-a-file-
http://www.perldoc.com/perl5.8.0/pod/perlopentut.html#File-Locking

 

[David Efflandt]

The seek to EOF is correct when you're appending to a file. In the
general case (and particularly when reading) it's not required.


The file pointer is on a per-filehandle basis, not on a per-file
basis. We'd get mighty upset if I changed my file pointer and yours
updated too.

Actually I had problems with a CGI script that just read a file without
flocking it. If more than one instance tried to read from the file at the
same time, it apparently got confused where the file pointer was and the
Perl (CGI) scripts would go into endless loops (the host notified me about
the endless processes). Flocking solved that. I think that was on
Solaris, which may do things differently than other Unix systems.

 

[news]

Actually I had problems with a CGI script that just read a file without
flocking it. If more than one instance tried to read from the file at the
same time, it apparently got confused where the file pointer was and the
Perl (CGI) scripts would go into endless loops (the host notified me about
the endless processes).

That'll be something to do with the way CGI processes have been
implemented in your web server.

Chris