Finding out if another copy of a CGI Perl scripts is running?

[Lisa]

How would one go about finding out if another copy of a CGI Perl
scripts is running?

I want to set up a script to run as a cron job, starting about once a
minute, however, the first thing I want to do when the script starts is
to check and see if the last incarnation is still (highly unlikly but
could happen on very busy days) running and shut down if it is, so that
there is only one version of the script proccessing data.

Lisa

 

[Brian McCauley]

How would one go about finding out if another copy of a CGI Perl
scripts is running?

Have it lock something.

I want to set up a script to run as a cron job,

Huh? I thought we were talking about CGI scripts, not cron scripts.
Not of course that it makes any difference.

starting about once a
minute, however, the first thing I want to do when the script starts is
to check and see if the last incarnation is still (highly unlikly but
could happen on very busy days) running and shut down if it is, so that
there is only one version of the script proccessing data.

In that case, the data would seem the obvious thing to lock.

 

[Alan Mead]

Star date: Sat, 18 Dec 2004 15:00:23 -0800, Lisa's log:

I want to set up a script to run as a cron job, starting about once a
minute, however, the first thing I want to do when the script starts is
to check and see if the last incarnation is still (highly unlikly but
could happen on very busy days) running and shut down if it is, so that
there is only one version of the script proccessing data.

First, if it runs from a cron job then it's just a script. CGI is a way
that scripts generate dynamic web pages.

There are probably several ways you can do this. An obvious one would be
to make a "lock file"... if your script detects the lock file then either
(1) there is a copy running or (2) a copy died leaving a stale
lock.

I've never done this in Perl, but I bet you could grep the running
processes for your script name (e.g., 'my $rs = `ps -aux | grep
[m]yscript.pl`'). Of course, you'd have to ignore the currently running
version.

 

[Lisa]

OK, but I do not want the second iteration to wait for a lock I want it
to recognize the existance of the previously running version still
being active and to exit gracefully.

 

[Gunnar Hjalmarsson]

OK, but I do not want the second iteration to wait for a lock I want it
to recognize the existance of the previously running version still
being active and to exit gracefully.

So don't have it wait, but have it exit if the file is locked.

 

[Alan Mead]

Star date: Sat, 18 Dec 2004 15:45:03 -0800, Lisa's log:

OK, but I do not want the second iteration to wait for a lock I want it
to recognize the existance of the previously running version still
being active and to exit gracefully.

exit if ( -x $lockfile );

'-x' is from memory...

 

[Alan J. Flavell]

First, if it runs from a cron job then it's just a script.
agreed

CGI is a way that scripts generate dynamic web pages.

CGI is a software interface between a web server and a script.

What you do with it beyond that is... well, limited only by the
designer's imagination...

I don't know about you, but to me the term "dynamic web pages" could
cover a wide multitude of sins. Many of which don't involve CGI.

But yes, you could dynamically create web pages. Creating "dynamic
web pages" isn't quite the same thing, though.

SCNR.

 

[KKramsch]

OK, but I do not want the second iteration to wait for a lock I want it
to recognize the existance of the previously running version still
being active and to exit gracefully.

Then request a non-blocking lock:

open MY_DATA, $datafile or die "Couldn't read $datafile\n";
exit GRACEFULLY unless flock(MY_DATA, LOCK_EX|LOCK_NB);

See The Perl Cookbook pp. 245-247.

Karl

 

[Lisa]

Here is what I have been trying and it does not work as I expexcted it
to.

if(-e "LockFile.txt"){
print "LockFile already exists",br();
exit;
}else{
if(open("LOCKFILE",">LockFile.txt")){
print "opening LockFile",br();
print LOCKFILE "1";
close LOCKFILE;
}else{
die "Can't open LockFile";
}
}

sleep(20)
unlink("LockFile.txt");

print "Done",br();
exit


If I comment out the sleep and unlink commands, and let the script end
without deleting the file, then the next run of the script finds the
file and exits. However if I leave the sleep unlink in the script and
run two iterations of the script they both run instead of the second
one exiting because the first one is sleeping and has not unlinked the
lock file as I expect it to.

Lisa

 

[Joe Smith]

I've never done this in Perl, but I bet you could grep the running
processes for your script name (e.g., 'my $rs = `ps -aux | grep
[m]yscript.pl`'). Of course, you'd have to ignore the currently running
version.

I have an extremely NON-portable version that works on Linux.
-Joe

#!/usr/bin/perl
# Purpose: Starts a command only if it is not already running

use strict; use warnings;

my $Usage = "Usage: $0 command args\n";
# Runs command if not already running
my @cmd = @ARGV or die $Usage;
my $cmd_line = "@cmd";
die "Command line too long: $cmd_line\n" if length $cmd_line > 4095;

my ($user,$header) = get_uid_from_ps();
my $count = 0;
foreach (`ps -efww`) { # This has a limit of 4096 bytes for CMD
my ($who,$pid,$cmd) = (split /\s+/,$_,7)[0,1,6];
next unless $who eq $user and $cmd =~ /\Q$cmd_line\E$/;
next if $pid == $$;
print STDERR " $user is running '$cmd_line'\n$header" unless $count++;
print STDERR $_;
}
print STDERR " count = $count\n" if $count > 1;
exit 1 if $count;
print "$cmd_line\n";
exec @cmd;

sub get_uid_from_ps {
my @ps = `ps -fp $$`;
my @cols = split ' ',$ps[0];
$_ = "UID PID PPID C STIME TTY TIME CMD";
die "'@cols' != '$_'" unless "@cols" eq $_;
@cols = split ' ',$ps[1];
($cols[0], $ps[0]);
}

 

[A. Sinan Unur]

Here is what I have been trying and it does not work as I expexcted it
to.

The code you posted does not compile. Please see the posting guidelines and
follow those to help us help you.

D:\Home>perl t.pl
syntax error at t.pl line 18, near ")
unlink"
Execution of t.pl aborted due to compilation errors.

if(-e "LockFile.txt"){
print "LockFile already exists",br();
exit;

This is wrong. The file can come into existence between the test and the
open below.

However if I leave the sleep unlink in the script and run two iterations
of the script they both run instead of the second one exiting because the
first one is sleeping and has not unlinked the lock file as I expect it
to.

This analysis is likely incorrect. At the very least, you should have read

perldoc -q lock
perldoc -f flock

before posting.

Other suggested reading:

http://tinyurl.com/6q2gc
http://tinyurl.com/6vzy6

Sinan.

 

[Chris Mattern]

Star date: Sat, 18 Dec 2004 15:45:03 -0800, Lisa's log:


exit if ( -x $lockfile );

'-x' is from memory...

Race condition. The testing & creation of the lockfile is not an
atomic function. One could have a situation where program A
tests for $lockfile's existence, finds it doesn't exist, is
interrupted, and then program B tests for $lockfile's existence,
finds it doesn't exist. Boom.

Granted, from the problem description is chances this will happen
in the OP's situation is about nil, but one should avoid race
conditions as general good programming practice. And one can
never be sure when the circumstances surrounding the lock might
change...

The proper way to do it is a lock *directory*, because mkdir
fails if the directory already exists.

exit if (! mkdir($lockfile));

This tests and acquires the lock in one kernel call.
--
Christopher Mattern

"Which one you figure tracked us?"
"The ugly one, sir."
"...Could you be more specific?"

 

[KKramsch]

Here is what I have been trying and it does not work as I expexcted it
to.

if(-e "LockFile.txt"){
print "LockFile already exists",br();
exit;
}else{
if(open("LOCKFILE",">LockFile.txt")){
print "opening LockFile",br();
print LOCKFILE "1";
close LOCKFILE;
}else{
die "Can't open LockFile";
}
}

print "Done",br();
exit

If I comment out the sleep and unlink commands, and let the script end
without deleting the file, then the next run of the script finds the
file and exits. However if I leave the sleep unlink in the script and
run two iterations of the script they both run instead of the second
one exiting because the first one is sleeping and has not unlinked the
lock file as I expect it to.

It works fine for me (once I fixed the missing ; after sleep(20)
and defined br):

% ./locking_script.pl & ; sleep 1; ./locking_script.pl &
[3] 9593
opening LockFile
[4] 10126
LockFile already exists
[4] - done ./locking_script.pl
% Done

[3] - done ./locking_script.pl
%

The printing of "Done" occurs only once.

Karl

 

[Alan J. Flavell]

On Sun, 19 Dec 2004, KKramsch wrote:

[..snip..]

It works fine for me

"For some small values of the term 'works' "

The method is fundamentally unsound. The proper way to do
locking is based on an /atomic/ operation.

The only alternative - and it's hard, very hard, to be sure you can
get it right - is some kind of iterative locking procedure. This one
had neither.

Merlyn's columns have covered the topic of locking several times: at
least one of the articles (though I can't put my fingers on the right
one just now) has covered a way of iteratively trying to get control
of an exclusive resource - backing-off if the attempt failed - and
finally reaching a state in which one participant can be confident
that they have exclusive control of the resource.

http://www.stonehenge.com/merlyn/columns.html

Some of those articles are quite old now and I'm sure the author would
agree there are some points on which the style could be updated; but
the fundamentals are sound, and one can learn a lot from them.
Especially when one's tempted once again to re-invent the wheel :-}

 

[Anno Siegel]

Star date: Sat, 18 Dec 2004 15:45:03 -0800, Lisa's log:


exit if ( -x $lockfile );

'-x' is from memory...

Wrong in all respects. Brian (with capital "B" was talking about
locking a file (a safe way of ensuring mutual exclusion), not of
using a lock file, which is an unsafe approximation if done naively,
and still suffers from stale locks if done right.

Also, you remember wrong about "-x", it tests for executability. Please
don't post information you *know* may be wrong. Look it up and get it
right.

Anno

 

[KKramsch]

Other suggested reading:

http://tinyurl.com/6q2gc

In this post you propose the following:

use Fcntl qwflock);

sub update {

# ...

open my $lock, '>', 'lockfile'
or die "Cannot open lockfile: $!";
flock $lock, LOCK_EX
or die "Cannot obtain exclusive lock on lockfile: $!";

# Do the updating etc.

}

One thing that confuses me about this code is that whether flock
succeeds or not, lockfile gets clobbered by the open. The lock is
"advisory"; it does not block the open. I think this may be why
the docs use sysopen with O_RDWR instead of open $fh, ">file":

use Fcntl;

# ...

sysopen my $lock, 'lockfile', O_RDWR|O_CREAT
or die "Can't open lockfile: $!\n";

# I suppose the sysopen above could be replaced with
# open my $lock, '+<lockfile'

flock $lock, LOCK_EX|LOCK_NB
or die "Can't obtain exclusive lock on lockfile: $!\n";

truncate $lock, 0
or die "Can't truncate lockfile: $!\n";

# etc.

close $lock
or die "Can't close lockfile: $!\n";

flock $lock, LOCK_UN
or die "Can't unlock lockfile: $!\n";



Karl

 

[Alan J. Flavell]

On Sun, 19 Dec 2004, KKramsch wrote:

[..]

One thing that confuses me about this code is that whether flock
succeeds or not, lockfile gets clobbered by the open.

In *this* particular scheme, the lockfile exists solely for the
purpose of getting a lock put on it. Its contents are irrelevant.

*If* one is trying to get an exclusive lock on an actual data file,
then it might be more useful and obvious if the lock was put on the
data file itself, than to create some auxiliary file to hold the lock.
And then you'd look for an alternative strategy for doing the open,
indeed.

The lockfile technique is more often used when trying to implement
exclusive access to some /other/ resource than a file.

Does that help a bit?

The lock is "advisory"; it does not block the open.
Correct.

I think this may be why
the docs use sysopen with O_RDWR instead of open $fh, ">file":

If you want to put a lock on a data file, then indeed that's
a good approach

sysopen my $lock, 'lockfile', O_RDWR|O_CREAT
or die "Can't open lockfile: $!\n";

OK, but this could be your actual data file.

Or perhaps open for append ('>>') , depending on what the process
intends to do with the file once it's got it.

# I suppose the sysopen above could be replaced with
# open my $lock, '+<lockfile'

If the file doesn't already exist, then that's going to fail. So,
unlike the other techniques, this one isn't self-starting - someone
has got to create an initial (maybe empty) file before the circus can
begin.

And of course all of the above are going to be reliant on just what
is supported by the particular OS. AIUI some are a bit more portable
between OSes than others.

hth

 

[A. Sinan Unur]

In this post you propose the following:


One thing that confuses me about this code is that whether flock
succeeds or not, lockfile gets clobbered by the open.

I should clarify. The lock file in the code above is meant to be used as a
sentinel to see if it is OK to update the actual data file one is
interested in. One would only open the data file in the

# Do the updating etc

section. Of course, the OP would check if the exclusive lock succeeded and
return from the sub if it did not.

Sinan.

 

[bill]

See "There can be only one! - The Highlander solution" at
<http://www.stonehenge.com/merlyn/WebTechniques/col54.html>.

which has the following:

=7= my $count = 0;
=8= {
=9= flock HIGHLANDER, LOCK_EX | LOCK_NB and last;
=10= sleep 1;
=11= redo if ++$count < 10;

Line 11 increments count, and ensures that it is still below 10.
If so, the redo operator pops back up to line 8, retrying the flock.
If not, we've tried 10 times to flock, or ur, actually, 9 times to
flock (durn fencepost off-by-one errors!), and it's time to report
the error.

Maybe I'm missing something but I think your code is right (no
off-by-one error: it tries flock at most 10 times). It's easy to
see if you change the "10" on line 11 to "1" or "2".

bill

 

[A. Sinan Unur]

I should clarify. The lock file in the code above is meant to be used
as a sentinel to see if it is OK to update the actual data file one is
interested in. One would only open the data file in the

# Do the updating etc

section. Of course, the OP would check if the exclusive lock succeeded
and return from the sub if it did not.

Probably by doing (untested):

flock $lock, LOCK_EX|LOCK_NB or return;

Please also see Randal Schwartz's post. After all, I learned a lot of what
little I know about Perl from his columns.

Sinan