Yet another flock question...

[kj]

For the following, assume the OS is some Un*x.

Suppose one uses flock like this (lifted with minor modifications
from perlfunc):

use Fcntl ':flock'; # import LOCK_* constants

sub lock {
flock(LOG,LOCK_EX);
# and, in case someone appended
# while we were waiting...
seek(LOG, 0, 2);
}

sub unlock {
flock(LOG,LOCK_UN);
}

open(LOG, ">>/var/log/my.log"
or die "Can't open logfile: $!";



# elsewhere...

lock();
print LOG $msg,"\n\n";
unlock();


....but suppose that after the open() but before lock(), some
other process (e.g. a logfile rotator) moves /var/log/my.log to
/var/log/my.log.1, creates a new /var/log/my.log file, and perhaps
a *third* process then writes some new stuff to this new /var/log/my.log.
(Note that this is not an improbable scenario; in a long-running
process, such as a server, ample time can occur between the executions
of the open() and lock() statements for all of this to happen).

In this case, the code above will end up appending $msg to
/var/log/my.log.1 and not to the newly created /var/log/my.log.
Now /var/log/my.log.1 would have entries that are more recent than
the earliest entries in /var/log/my.log, which can lead to problems
or at least confusion.

How could the code above be modified to ensure that not only the
print statement is on an exclusively-locked handle, but also that
the file being written to is indeed the latest instance of
/var/log/my.log?

I can't think of any solution. The best I can come up (which sucks)
is to always open the handle immediately before requesting the lock
(i.e. no intervening statements between the open() and the lock()).
All the external events between the open() and the lock() described
above can still happen, but at least now the probability of this
is greatly reduced.

There's got to be something better, but I'm drawing a blank. Any
help would be appreciated.

TIA!

kj

 

[Mark Clements]

For the following, assume the OS is some Un*x.

Suppose one uses flock like this (lifted with minor modifications
from perlfunc):

use Fcntl ':flock'; # import LOCK_* constants

sub lock {
flock(LOG,LOCK_EX);
# and, in case someone appended
# while we were waiting...
seek(LOG, 0, 2);
}

sub unlock {
flock(LOG,LOCK_UN);
}

open(LOG, ">>/var/log/my.log"
or die "Can't open logfile: $!";



# elsewhere...

lock();
print LOG $msg,"\n\n";
unlock();


...but suppose that after the open() but before lock(), some
other process (e.g. a logfile rotator) moves /var/log/my.log to
/var/log/my.log.1, creates a new /var/log/my.log file, and perhaps
a *third* process then writes some new stuff to this new /var/log/my.log.
(Note that this is not an improbable scenario; in a long-running
process, such as a server, ample time can occur between the executions
of the open() and lock() statements for all of this to happen).

In this case, the code above will end up appending $msg to
/var/log/my.log.1 and not to the newly created /var/log/my.log.
Now /var/log/my.log.1 would have entries that are more recent than
the earliest entries in /var/log/my.log, which can lead to problems
or at least confusion.

How could the code above be modified to ensure that not only the
print statement is on an exclusively-locked handle, but also that
the file being written to is indeed the latest instance of
/var/log/my.log?

I can't think of any solution. The best I can come up (which sucks)
is to always open the handle immediately before requesting the lock
(i.e. no intervening statements between the open() and the lock()).
All the external events between the open() and the lock() described
above can still happen, but at least now the probability of this
is greatly reduced.

There's got to be something better, but I'm drawing a blank. Any
help would be appreciated.

This isn't a direct answer, but if this is a real-world problem rather
than a learning exercise, how about going through syslog and/or using a
logging package eg Log::Log4perl?

Mark

 

[xhoster]

....

...but suppose that after the open() but before lock(), some
other process (e.g. a logfile rotator) moves /var/log/my.log to
/var/log/my.log.1, creates a new /var/log/my.log file,

Also suppose this happens after the open and after the lock? What
would be the difference?

and perhaps
a *third* process then writes some new stuff to this new /var/log/my.log.
(Note that this is not an improbable scenario; in a long-running
process, such as a server, ample time can occur between the executions
of the open() and lock() statements for all of this to happen).

In this case, the code above will end up appending $msg to
/var/log/my.log.1 and not to the newly created /var/log/my.log.
Now /var/log/my.log.1 would have entries that are more recent than
the earliest entries in /var/log/my.log, which can lead to problems
or at least confusion.

Since you know about this possibility, then take steps to avoid those
problems or confusion. That is the way I'd handle it.

How could the code above be modified to ensure that not only the
print statement is on an exclusively-locked handle, but also that
the file being written to is indeed the latest instance of
/var/log/my.log?

I can't think of any solution. The best I can come up (which sucks)
is to always open the handle immediately before requesting the lock
(i.e. no intervening statements between the open() and the lock()).
All the external events between the open() and the lock() described
above can still happen, but at least now the probability of this
is greatly reduced.

The rotator program could take out an exlcusive lock on the file before
renaming, and hold that lock for (say) 10 minutes after renaming it.
When your logger obtains a lock, it would always check to make sure the
lock was not on a file open for more than 10 minutes. And when tries to
obtain a lock it initially does so non-blockingly, and if it fails it
reopens the file and tries again.

Or you could not rename files. Have the rotator copy the data and then
truncate the original, rather than renaming it.

Xho

 

[kj]

OK, I figured it out (in fact I've seen this before elsewhere, but
I'd forgotten): one locks a sentinel file (*not* the log file) that
one expects won't be renamed or deleted. Once we have the lock,
we can test whether the handle to the log remains valid, and make
adjustments if not.

kj