flock not locking

[xargle]

I'm using perl 5.8.5 under RHEL 4, and trying to produce a quick
sockets server with 240 forked processes all accepting from a listener
socket. Which fork gets to accept is governed by who gets the flock,
which would normally be a flock on the listener socket, but as that
didn't work I tried using a file in /tmp and flocking that. This is
apparently what apache does. This is an attempt at optimising an
existing application to reduce load as connections to the server are
very short and very regular, so the constant forking is undesirable...

From the entry point the code looks like this :

my $kid;
my $socket = IO::Socket::INET->new(
LocalPort => '4576',
Proto => 'tcp',
Listen => SOMAXCONN,
Reuse => 1)
or die "Failed to listen: $!";
my $i=0;

# ignore the kids, they're noisy little gits
$SIG{CHLD} = 'IGNORE';

# open a temp file to provide us with a locking mechanism as
# perl doesn't appear to like us locking socket handles
sysopen(LOCKFILE, "/tmp/test.lock", O_WRONLY | O_CREAT) || die "Can't
open lock file /tmp/test.lock\n";

STDOUT->autoflush(1);

while ($i < $tofork)
{
sleep 1;
$i++;
# fork here, if we're the parent just loop
next if $kid = fork;

print STDOUT "Forked worker on pid " . getpid() . "\n";

while(1)
{
print STDOUT getpid() . " requesting LOCK_EX\n";
flock LOCKFILE,2 or die "cant flock: $!"; # 2 = LOCK_EX, exclusive
lock
print STDOUT getpid() . " has gained LOCK_EX\n";
my $client = $socket->accept();
print STDOUT getpid() . " accepted connection from " .
$client->sockhost . " and will unlock listener\n";
flock LOCKFILE,8 or die "cant flock: $!"; # 8 = LOCK_UN, unlock

process_connection($client);
print STDOUT getpid() . " session is done\n";
}

while(1)
{
sleep(60);
}
}

The while(1) sleep is a desperate measure whilst attempting to get the
locking working in case the parent thread exiting was the issue, ditto
the sleep 1 between starting the threads.

What happens is the 240 threads start, but the flock LOCK_EX (2)
doesn't block at all and returns no error, so all the forks end up
trying to accept which is precisely what I don't want to occur.

Am I missing something here? Does flock work in 5.8.5? Is it something
to do with the forking?

Any advice would be greatly appreciated.

 

[xhoster]

I'm using perl 5.8.5 under RHEL 4, and trying to produce a quick
sockets server with 240 forked processes all accepting from a listener
socket. Which fork gets to accept is governed by who gets the flock,
which would normally be a flock on the listener socket, but as that
didn't work I tried using a file in /tmp and flocking that. This is
apparently what apache does. This is an attempt at optimising an
existing application to reduce load as connections to the server are
very short and very regular, so the constant forking is undesirable...
....

# open a temp file to provide us with a locking mechanism as
# perl doesn't appear to like us locking socket handles
sysopen(LOCKFILE, "/tmp/test.lock", O_WRONLY | O_CREAT) || die "Can't
open lock file /tmp/test.lock\n";

STDOUT->autoflush(1);

while ($i < $tofork)
{
sleep 1;
$i++;
# fork here, if we're the parent just loop
next if $kid = fork;

Each child has to open the tmp file anew, to get an independent handle onto
the file. When you open the file before forking, every forked process is
sharing the same handle, and thus they can't lock each other out.

So just move the sysopen into the loop, after the fork.

Xho

 

[Ben Morrow]

Quoth (e-mail address removed):

I'm using perl 5.8.5 under RHEL 4, and trying to produce a quick
sockets server with 240 forked processes all accepting from a listener
socket. Which fork gets to accept is governed by who gets the flock,
which would normally be a flock on the listener socket, but as that
didn't work I tried using a file in /tmp and flocking that. This is
apparently what apache does. This is an attempt at optimising an
existing application to reduce load as connections to the server are
very short and very regular, so the constant forking is undesirable...

Further to what Xho said (each child needs to open the file itself),
here is some more advice:

my $kid;

Don't declare variables before you need them.

my $socket = IO::Socket::INET->new(
LocalPort => '4576',
Proto => 'tcp',
Listen => SOMAXCONN,
Reuse => 1)
or die "Failed to listen: $!";
my $i=0;

Variables are automatically initialized to undef, which is 0 in numeric
context. However see below...

# ignore the kids, they're noisy little gits
$SIG{CHLD} = 'IGNORE';

# open a temp file to provide us with a locking mechanism as
# perl doesn't appear to like us locking socket handles

It's not perl, it's the OS. On most OSen, sockets don't accept locks.

sysopen(LOCKFILE, "/tmp/test.lock", O_WRONLY | O_CREAT) || die "Can't
open lock file /tmp/test.lock\n";

It's better to use lexical filehandles. These work with sysopen as well
as open

sysopen (my $LOCKFILE, '/tmp/test.lock', O_WRONLY|O_CREAT)
or die "can't open lock file /tmp/test.lock: $!\n";

I've kept your parens, but if you use or instead of || you can drop
them.

STDOUT->autoflush(1);

I *really* don't like this. STDOUT is not an object, even though it
pretends to be. Learn how to use the special variables.

while ($i < $tofork)

This is just an unwrapped C-style for loop, without the subtlety of
using a continue block to do the increment. It is better written in Perl
as

for (1..$tofork) {

{
sleep 1;
$i++;
# fork here, if we're the parent just loop
next if $kid = fork;

print STDOUT "Forked worker on pid " . getpid() . "\n";

The current pid is available as $$.

while(1)
{
print STDOUT getpid() . " requesting LOCK_EX\n";
flock LOCKFILE,2 or die "cant flock: $!"; # 2 = LOCK_EX, exclusive
lock

Don't *ever* hardcode flock values. Get them from the Fcntl module:

use Fcntl qw/:flock/;

flock LOCKFILE, LOCK_EX;

print STDOUT getpid() . " has gained LOCK_EX\n";
my $client = $socket->accept();
print STDOUT getpid() . " accepted connection from " .
$client->sockhost . " and will unlock listener\n";
flock LOCKFILE,8 or die "cant flock: $!"; # 8 = LOCK_UN, unlock

This may have been one of the rare cases that an explicit LOCK_UN wasn't
a bug, but as you have to reopen the file to get a new lock anyway
you're better off closing it to unlock. If you use lexical FHs you just
need to scope the variable over the piece of code you want under the
lock. As an added bonus if you ever move to threads instead (not
something I'd recommend with the current Perl thread model under Unix)
you won't need to change much.

You may also want to look at the modules on CPAN which do this, such as
Parallel::ForkManager or Proc::Fork.

Ben

 

[xargle]

Thanks Xho and Ben, Xho was spot on in identifying the issue, and I've
corrected all the problems Ben pointed out. I've been writing C for
over a decade so certain styles have become habit which don't
necessarily apply under perl

Nice to see perlites are so helpful!

 

[xhoster]

Quoth (e-mail address removed):

Don't *ever* hardcode flock values. Get them from the Fcntl module:

use Fcntl qw/:flock/;

flock LOCKFILE, LOCK_EX;


This may have been one of the rare cases that an explicit LOCK_UN wasn't
a bug, but as you have to reopen the file to get a new lock anyway
you're better off closing it to unlock.

I don't agree with you in this specific context. He only needs to open the
file once per process, i.e. once per fork. He can simply lock and unlock
it inside the accept loop (or at least I see no reason he can't do it that
way). There is no need to (re)open the file after every accept, only after
every fork.

Xho

 

[Ben Morrow]

Quoth (e-mail address removed):

I don't agree with you in this specific context. He only needs to open the
file once per process, i.e. once per fork. He can simply lock and unlock
it inside the accept loop (or at least I see no reason he can't do it that
way). There is no need to (re)open the file after every accept, only after
every fork.

You are right (of course ). However, there's no *harm* in reopening
the file either (as each child opens it individually, the problem of
someone moving the file out from under you is not being dealt with
anyway), and I at least find variable scope a natural way of dealing
with resources that need freeing.

Ben