Script continues running after an exit() call?

[Debo]

I realize this post is going to be noninformative, but the script I'm
writing is rather long and so in the interest of brevity I'd like to just
summarize the problem I'm having.

I've written a script that, in general terms:

1. Opens a log file for writing, using Log:Log4Perl

1. Uses system() to call a program (NCBI BLAST) to associate the data in
two files. BLAST writes out a report of its activity. This report is quite
large (~600 megs).

2. Opens a handle to a postgres database (with AutoCommit off) using DBI.

3. Creates a cached INSERT query, and then parses the report, using this
query to insert lots (~20000) records based on this report. Some of the
insertions fail, but that doesn't really concern me because I sort of
expect the process to be lossy. (I don't know if this screws up the
ability to commit the changes).

4. Commits the changes to the database.

5. Uses the move() method of File::Copy to move the log file to a
different location.

6. Calls exit(0)

Now, here's the rub. The script will sometimes succeed or sometimes fail
to commit, which I think I can fix without much difficulty. However, when
I run this overnight and return in the morning, I INVARIABLY find about
18-20 perl processes running, and the BLAST command still running.

I look at the log that the script generates (in its new location, after
the move() call), and it seems that after the script reaches the exit(0),
it starts the BLAST process again, and continues to do so repeatedly. In
all, BLAST will be called and will complete execution about 3 or 4 times
throughout the course of the night, when the flow of control in the script
quite obviously shows that it should only execute once.

The thing I find especially strange is that the script continues to log
changes to the log file *after it has been moved*! It does this regardless
of whether I use the logger, or a plain filehandle to do my logging. The
script does not output anything pertaining to this problem on stderr (it's
mostly just some gunk that bioperl spits out for no particular reason).

So, I suppose my questions are:

1) Under what circumstances would perl be enticed to spawn a legion of
subprocesses without any explicit fork etc calls in the script?

2) How could these processes continue to write to a file after it has been
moved?

3) Why would a script 'ignore' an exit(0) call?

My platform right now is cygwin (shudder) with perl v5.8.5. I'd love
to be doing this in linux, but external forces have imposed the
environment and now i'm stuck with it.

Any tips you could provide that would help me diagnose the problem would be
appreciated.

Thanks!

-Debo

 

[Jim Keenan]

I realize this post is going to be noninformative, but the script I'm
writing is rather long and so in the interest of brevity I'd like to just
summarize the problem I'm having.

Since there are at least a half-dozen places where your script could be
going awry, I can't give anything like a definitive answer. But, if
you haven't done so already, I would populate the script with calls to
"die" with appropriate error messages and see how far the script gets
when certain conditions are not met.

jimk

 

[xhoster]

The thing I find especially strange is that the script continues to log
changes to the log file *after it has been moved*! It does this
regardless of whether I use the logger, or a plain filehandle to do my
logging.

On unix/linux, moving a file (within a file system) is little more than
a change in name. It doesn't effect any already-open handles to that file.

The script does not output anything pertaining to this problem
on stderr (it's mostly just some gunk that bioperl spits out for no
particular reason).

I'd suggest you liberally sprinkle your code with "warn" statements.

warn "$$ I'm about to start BLAST";
#...
warn "$$ I've just finished BLAST";
#...
warn "$$ I'm about to exit";

So, I suppose my questions are:

1) Under what circumstances would perl be enticed to spawn a legion of
subprocesses without any explicit fork etc calls in the script?

You are probably mis-interpreting something.

3) Why would a script 'ignore' an exit(0) call?

You are probably mis-interpreting something.
You might want to try POSIX::_exit, though I doubt that will
be the problem.

Xho

 

[kevindotcar]

I realize this post is going to be noninformative, but the script I'm
writing is rather long and so in the interest of brevity I'd like to just
summarize the problem I'm having.

I've written a script that, in general terms:

1. Opens a log file for writing, using Log:Log4Perl
[---]

5. Uses the move() method of File::Copy to move the log file to a
different location.

6. Calls exit(0)

Hmmm- do you close the logfile before doing the move?

[---]

Thanks!

-Debo

HTH-
kDot