What with this open file descriptor/"Read on closed filehandle " stuff?

  • Thread starter Rex Gustavus Adolphus
  • Start date
R

Rex Gustavus Adolphus

Hej hej

This
I recently got problems with this (rather ugly, it was one of my
first) subroutine:

sub check_for_file {
my ($dbh, $filename,$wait_after_last_write, $msglevel) = @_;
my ($file);
my @files;
my $age_in_seconds;

&printlog($msglevel,3,"looking for $filename");
@files = glob($filename);
FILENAME: foreach $file (@files) {
next FILENAME unless -s $file; # next iteration of the
loop unless file exists
$age_in_seconds = time() - (stat($file))[9];
&printlog($msglevel,3,"Found $file, last changed
$age_in_seconds seconds ago."
. " Should be older than
$wait_after_last_write seconds");

return($file) if ($age_in_seconds >=
$wait_after_last_write);
}
return;
}

On a couple of rare, unrepeatable instances, this has caused the
following:
Read on closed filehandle <_GEN_0> at xxxx line 39, <_GEN_0> chunk
yyyyy.
where line 39 is the line with the call to glob.

I'm not at that site now, but it's an older perl, 5.005, I think,
running on Solaris, (sorry for being so vague).

I've done a bit of research and found this thread from last century:
http://groups.google.com/groups?hl=...&q=read+on+closed+filehandle+glob&sa=N&tab=wg

This seems to be just like my case.

I didn't think my code opened any file descriptors at all.

In other parts of the program I am renaming or moving the found file
with this code:
sub move_file {
my ($dbh, $to_dir, $file, $msglevel) = @_;
my $target = $to_dir;
if ( -d $to_dir ) {
$target .= "/".basename($file);
}

&printlog($msglevel,3,"moving $file to $target");
if ( (rename $file => $target ) or move($file,$target) ) {
return $target;
} else {
printlog($msglevel,0,"rename $file to $target failed, system
message:$!");
rename $file => $file.time()
or die "$0 terminates due to problems moving
files:\n$0 $!\n";
return;
}
} # end of sub move_file



So finally my questions:
Does glob open file descriptors for the found files?
Does rename or File::move open file descriptors?
Would using File::glob solve the issue?

Thanx for reading so far.
 
B

Ben Morrow

sub check_for_file {
my ($dbh, $filename,$wait_after_last_write, $msglevel) = @_;
my ($file);
my @files;
my $age_in_seconds;

You don't really need to declare these up here: it's better to keep
variables in as small a scope as possible, and simply declare them the
first time you use them.
&printlog($msglevel,3,"looking for $filename");

Don't call subs with & unless you need to (here you don't).
@files = glob($filename);
FILENAME: foreach $file (@files) {
next FILENAME unless -s $file; # next iteration of the
loop unless file exists
$age_in_seconds = time() - (stat($file))[9];

You might consider using File::stat to make this clearer.
&printlog($msglevel,3,"Found $file, last changed
$age_in_seconds seconds ago."
. " Should be older than
$wait_after_last_write seconds");

return($file) if ($age_in_seconds >=
$wait_after_last_write);
}
return;
}

On a couple of rare, unrepeatable instances, this has caused the
following:
Read on closed filehandle <_GEN_0> at xxxx line 39, <_GEN_0> chunk
yyyyy.
where line 39 is the line with the call to glob.

In other parts of the program I am renaming or moving the found file
with this code: ....
if ( (rename $file => $target ) or move($file,$target) ) {

There's no need to do that (I take it move() comes from File::Copy?) as
it will use rename if it can.
I'm not at that site now, but it's an older perl, 5.005, I think,
running on Solaris, (sorry for being so vague).
I didn't think my code opened any file descriptors at all.

So finally my questions:
Does glob open file descriptors for the found files?

On older perls (pre-5.6), glob is performed by forking a csh(1). This is
a Bad Thing, so it was replaced with File::Glob in 5.6.
Does rename or File::move open file descriptors?
^^ Copy::, almost certainly
rename: no.
File::Copy::move: it depends on whether it could be done with rename or
if the file had to be copied then deleted.
Would using File::glob solve the issue?
^ G
Yes.

Ben
 
R

Rex Gustavus Adolphus

Thanks for your valuable input, Ben!
I appreciate it!
A few follow-up-questions and comments below:

Ben Morrow said:
You don't really need to declare these up here: it's better to keep
variables in as small a scope as possible, and simply declare them the
first time you use them.

I guess this is mostly a matter of personal taste (TMTOWTDI), or is
there any special reason your way is better?
Don't call subs with & unless you need to (here you don't).

Isn't this another case of TMTOWTDI?


There's no need to do that (I take it move() comes from File::Copy?) as
it will use rename if it can.

Yes, of course, I'll change that!

On older perls (pre-5.6), glob is performed by forking a csh(1). This is
a Bad Thing, so it was replaced with File::Glob in 5.6.

But even if it is performed that way, why is the program running out
of file descriptors eventually?
Seems to me like a bug in the pre-5.6-glob() or File::Copy::move then.
If glob() or File::Copy::move opens filedescriptors they should close
them to, or??
^^ Copy::, almost certainly
rename: no.
File::Copy::move: it depends on whether it could be done with rename or
if the file had to be copied then deleted.

^ G
Yes.

But File::Glob isn't available right now (pre-5.6 perl as it is),
so I figure I could do it this way:

I forgot to mention it in my first post, but more often than not
there's
actually no need to glob() at all since $filename seldom contains
wildcards.

I change this code
@files = glob($filename);
to this:
$_ = $filename;
# check for existence of ? and/or *
if ( m/.*[\?|\*].*/ ) {
@files = glob($filename);
} else {
push(@files,$filename);
}

It's not a perfect solution, but will at least make the number of
glob()-call much less.
I guess it won't help much if the real problem is the
File::Copy::move-call

/RGA
 
T

Tassilo v. Parseval

Also sprach Rex Gustavus Adolphus:
I guess this is mostly a matter of personal taste (TMTOWTDI), or is
there any special reason your way is better?

Fewer variables are always a good thing. By putting them into the
smallest possible scope, variables will only be accessible therein and
not somewhere else.

Also, perl will destroy them more timely and thus can reuse the memory
associated with them.
Don't call subs with & unless you need to (here you don't).

Isn't this another case of TMTOWTDI?
Yes.

But File::Glob isn't available right now (pre-5.6 perl as it is),
so I figure I could do it this way:

I forgot to mention it in my first post, but more often than not
there's
actually no need to glob() at all since $filename seldom contains
wildcards.

I change this code
@files = glob($filename);
to this:
$_ = $filename;
# check for existence of ? and/or *
if ( m/.*[\?|\*].*/ ) {
@files = glob($filename);
} else {
push(@files,$filename);
}

It's not a perfect solution, but will at least make the number of
glob()-call much less.

Be aware however, that your regex is too simpleminded. It doesn't catch
(for instance):

file.[ch]

whereas it (falsely) does catch

file\*.c

In the latter, the asterics is escaped and thus to be taken as a literal
literally.

Tassilo
 
M

Matt Garrish

Tassilo v. Parseval said:
Also sprach Rex Gustavus Adolphus:


Yes.

I have to ask why you keep going against the grain on this? The & obviously
has a special meaning in subroutine calls, whether it's applicable in this
situation or not. Even if it never causes a problem for you to prefix your
subroutine calls with an ampersand, it's still not good practice (and goes
beyond a mere style issue). By saying that it's just another way to call a
subroutine you are implying that there is no difference, and that's patently
untrue.

Matt
 
T

Tassilo v. Parseval

Also sprach Matt Garrish:
I have to ask why you keep going against the grain on this? The & obviously
has a special meaning in subroutine calls, whether it's applicable in this
situation or not. Even if it never causes a problem for you to prefix your
subroutine calls with an ampersand, it's still not good practice (and goes
beyond a mere style issue). By saying that it's just another way to call a
subroutine you are implying that there is no difference, and that's patently
untrue.

Of the two side-effects that exist (passing @_ and circumventing the
prototype), only the latter could ever come into the game as the
parameters were passed explicitely. What remains are prototypes that
weren't used in this case.

When speaking about prototypes, we have to acknowledge that they
themselves are a rather questionable concept. They were originally
introduced in order to create functions behaving similar to some
builtins (such as push() or map()). I don't see why the ampersand should
be condemned just because they affect something that is used in maybe 1%
of all function definitions in existing Perl code.

Finally the statement

Don't call subs with & unless you need to.

is a violation against anything that Perl stands for. After all, we
don't object when people write object-oriented Perl although you never
'need to' and we all know that a simple method call is by a magnitude
slower than calling a plain function. The above statement implies that
something like a correct way of doing things exists. But it doesn't.
Neither in Perl nor in programming in general.

Having said that, I wouldn't mind if people explained what the ampersand
does and pointed out in which circumstances it could be wrong. That
would put someone into the position of deciding whether he wants it or
not. Simply telling not to use it is the wrong thing.

Tassilo
 
B

Ben Morrow

[please wrap attribution lines at 72 columns]

I guess this is mostly a matter of personal taste (TMTOWTDI), or is
there any special reason your way is better?

It is not so much a matter of taste as a matter of good programming
practice. If variables are declared in the smallest scope possible
there is the minimal chance of confusion with other variables of the
same name. Also, it is then (usually) possible to see at a glance the
whole scope the variable is valid over.
Isn't this another case of TMTOWTDI?

No, the two different ways of calling subs have different semantics (see
perlsub). Even if it makes no difference in this case, it will confuse
someone else reading your code (they will spend time trying to work out
why you needed the &-semantics).
But even if it is performed that way, why is the program running out
of file descriptors eventually?
Seems to me like a bug in the pre-5.6-glob() or File::Copy::move then.

Yes, it does; unless you are performing many globs simultaneously?
But File::Glob isn't available right now (pre-5.6 perl as it is),
so I figure I could do it this way:

But you can install File::Glob from CPAN and use that. It is a much
better solution. (See the FAQ if you don't have root to install new
modules globally.)
I forgot to mention it in my first post, but more often than not
there's
actually no need to glob() at all since $filename seldom contains
wildcards.

I change this code
@files = glob($filename);
to this:
$_ = $filename;
# check for existence of ? and/or *
if ( m/.*[\?|\*].*/ ) {
@files = glob($filename);
} else {
push(@files,$filename);
}

It's not a perfect solution, but will at least make the number of
glob()-call much less.

There is still a bug waiting to bite, though.
I guess it won't help much if the real problem is the
File::Copy::move-call

Just for the hell of it, you could try installing the latest version of
File::Copy from CPAN as well.

Ben
 
R

Rex Gustavus Adolphus

Tassilo v. Parseval said:
Also sprach Rex Gustavus Adolphus:
I forgot to mention it in my first post, but more often than not
there's
actually no need to glob() at all since $filename seldom contains
wildcards.

I change this code
@files = glob($filename);
to this:
$_ = $filename;
# check for existence of ? and/or *
if ( m/.*[\?|\*].*/ ) {
@files = glob($filename);
} else {
push(@files,$filename);
}

It's not a perfect solution, but will at least make the number of
glob()-call much less.

Be aware however, that your regex is too simpleminded. It doesn't catch
(for instance):

file.[ch]

whereas it (falsely) does catch

file\*.c

In the latter, the asterics is escaped and thus to be taken as a literal
literally.

Yes, you're correct of course.
Someone must have solved this foolproof before.
I guess it's not as simple as requiring the character before * (or ?)
is not a backslash since this should be catched, or?:

file\\*.c
 
M

Matt Garrish

Tassilo v. Parseval said:
Also sprach Matt Garrish:
Of the two side-effects that exist (passing @_ and circumventing the
prototype), only the latter could ever come into the game as the
parameters were passed explicitely. What remains are prototypes that
weren't used in this case.

When speaking about prototypes, we have to acknowledge that they
themselves are a rather questionable concept. They were originally
introduced in order to create functions behaving similar to some
builtins (such as push() or map()). I don't see why the ampersand should
be condemned just because they affect something that is used in maybe 1%
of all function definitions in existing Perl code.

I don't disagree with you in principle, but you're still ackowledging that
there are occasions where things may not happen as one expects when using an
ampersand. The poster believed that it was just a style issue and Ben
pointed out that this wasn't true, with the sound advice that one shouldn't
use the & unless one needs to. You then countered to the effect that this
was not true. Consequently, you leave the impression that it is just a style
issue. It may only be < 1% of code that will ever be effected, but why not
start someone off on a safer footing (i.e., have them discover the use of
the ampersand when they need to pass the same @_ on, not by accident).

I get the feeling that you're looking at the situation from the perspective
of an experienced Perl programmer, in which case you don't need/want to be
lectured on ampersands and their effects and just want to be left alone to
use them (which is certainly your right). In the interests of promoting good
programming techniques, however, I can't agree that the same hands-off
approach should be taken with someone new to the language.
Finally the statement

Don't call subs with & unless you need to.

is a violation against anything that Perl stands for.

You're thinking of PERL. Perl stands for nothing... : )

Matt
 
T

Tassilo v. Parseval

Also sprach Matt Garrish:
I don't disagree with you in principle, but you're still ackowledging that
there are occasions where things may not happen as one expects when using an
ampersand. The poster believed that it was just a style issue and Ben
pointed out that this wasn't true, with the sound advice that one shouldn't
use the & unless one needs to. You then countered to the effect that this
was not true. Consequently, you leave the impression that it is just a style
issue. It may only be < 1% of code that will ever be effected, but why not
start someone off on a safer footing (i.e., have them discover the use of
the ampersand when they need to pass the same @_ on, not by accident).

I get the feeling that you're looking at the situation from the perspective
of an experienced Perl programmer, in which case you don't need/want to be
lectured on ampersands and their effects and just want to be left alone to
use them (which is certainly your right). In the interests of promoting good
programming techniques, however, I can't agree that the same hands-off
approach should be taken with someone new to the language.

I admit that I fell into a similar trap as Ben did in that I stated
something without any explanation. This is something I do very seldom. I
did it here because over the past few weeks this "don't use the
ampersand" has become a sort of mantra in this group. I get easily
annoyed by repetitive sayings and so I had to say something (in parts
for the sake of contradiction, I wont deny it).

Tassilo
 
R

Rex Gustavus Adolphus

Hi Matt and Tassilo!

Even if we're now deviating from my original question I like to
comment some of your input.

Matt Garrish said:
I don't disagree with you in principle, but you're still ackowledging that
there are occasions where things may not happen as one expects when using an
ampersand. The poster believed that it was just a style issue and Ben
pointed out that this wasn't true, with the sound advice that one shouldn't
use the & unless one needs to. You then countered to the effect that this
was not true. Consequently, you leave the impression that it is just a style
issue. It may only be < 1% of code that will ever be effected, but why not
start someone off on a safer footing (i.e., have them discover the use of
the ampersand when they need to pass the same @_ on, not by accident).

I get the feeling that you're looking at the situation from the perspective
of an experienced Perl programmer, in which case you don't need/want to be
lectured on ampersands and their effects and just want to be left alone to
use them (which is certainly your right). In the interests of promoting good
programming techniques, however, I can't agree that the same hands-off
approach should be taken with someone new to the language.

I get the same feeling, Matt, that you're looking at the situation
from the experienced Perl programmer perspective to.
I'm not very experienced in Perl, and so I never heard of the
different effects using/not using & in sub-calls.
A simple example would be great!
You're thinking of PERL. Perl stands for nothing... : )

What's PERL, Matt, ;) ?
quotes from perlfaq:
"Larry now uses "Perl" to signify the language proper and "perl" the
implementation of it, i.e. the current interpreter." +
"But never write "PERL", because perl is not an acronym, apocryphal
folklore and post-facto expansions notwithstanding."


RGA
 
R

Rex Gustavus Adolphus

Ben Morrow said:
[please wrap attribution lines at 72 columns]

Sorry, english is not my first language, what's an "attribution line"?
If it's the "References: <c70a85ff.0402200341..."-stuff at the start
of my posts,
it's nothing I can help, Google adds that when I post (I'm using
Google to read/post usenet).
It is not so much a matter of taste as a matter of good programming
practice. If variables are declared in the smallest scope possible
there is the minimal chance of confusion with other variables of the
same name. Also, it is then (usually) possible to see at a glance the
whole scope the variable is valid over.

OK.
A related question, is it in any way inefficient to redeclare
variables,
as in ex 2 below:
ex 1:
my $foo;
while ($this_is_true) {
$foo = 0;
...
# do fancy stuff
...
$foo = $something;
}

ex 2:
while ($this_is_true) {
my $foo = 0;
...
# do fancy stuff
...
$foo = $something;
}
No, the two different ways of calling subs have different semantics (see
perlsub). Even if it makes no difference in this case, it will confuse
someone else reading your code (they will spend time trying to work out
why you needed the &-semantics).

OK, thanks, I just found the very useful www.perldoc.com-site,
I'll read up on perlsub there,
I was not aware of the differences
Yes, it does; unless you are performing many globs simultaneously?
No, my program doesn't. Just one glob at a time.
But you can install File::Glob from CPAN and use that. It is a much
better solution. (See the FAQ if you don't have root to install new
modules globally.)



There is still a bug waiting to bite, though.

But fewer bugbites are better than many, in my opinion ... ;)
Just for the hell of it, you could try installing the latest version of
File::Copy from CPAN as well.

Ben

Actually, I convinced the systems owner to install and use 5.8.0,
I'll see what happens then.
 
B

Ben Morrow

Ben Morrow said:
[please wrap attribution lines at 72 columns]

Sorry, english is not my first language, what's an "attribution line"?
If it's the "References: <c70a85ff.0402200341..."-stuff at the start
of my posts,
it's nothing I can help, Google adds that when I post (I'm using
Google to read/post usenet).

in the header (the bit above the first blank line) you need not worry
about; the stuff in the body (the rest of your article) should be
wrapped at 72 columns.
OK.
A related question, is it in any way inefficient to redeclare
variables,

Don't even ask. You should not be thinking about efficiency over clarity
until you have written your program and found it to be too slow; then
you should profile it and only optimize the bits that are actually
slowing it down.
But fewer bugbites are better than many, in my opinion ... ;)

Well, yes; but when there is a solution with no bugs (use File::Glob)
this is better still.

Ben
 
P

Paul Lalli

OK.
A related question, is it in any way inefficient to redeclare
variables,
as in ex 2 below:
ex 1:
my $foo;
while ($this_is_true) {
$foo = 0;
...
# do fancy stuff
...
$foo = $something;
}

ex 2:
while ($this_is_true) {
my $foo = 0;
...
# do fancy stuff
...
$foo = $something;
}

Efficiency isn't the issue here. These two programs are doing something
different. The first one is declaring $foo's scope to be the whole file;
the second one's scope is only within the while loop. In other words, if
you put
print "$foo\n";
after the } that closes the while loop, the first script will print the
value $foo had at the last iteration of the while. The second script will
print the undef value (or, if you've enabled strictures, will give you a
compile-time error.

Make sure you know what you're telling your script to do before you worry
about how efficient it is.

Paul Lalli
 
R

Rex Gustavus Adolphus

Ben Morrow said:
No, the two different ways of calling subs have different semantics (see
perlsub). Even if it makes no difference in this case, it will confuse
someone else reading your code (they will spend time trying to work out
why you needed the &-semantics).

Well, now I looked in perlsub.
Quoting:
"
NAME(LIST); # & is optional with parentheses.
NAME LIST; # Parentheses optional if predeclared/imported.
&NAME(LIST); # Circumvent prototypes.
&NAME; # Makes current @_ visible to called subroutine.
"
and:
"A subroutine may be called using an explicit & prefix. The & is
optional in modern Perl..."

The second quote seems to contradict the first partly.

The first I dont understand fully.
I think I understand
&NAME; # Makes current @_ visible to called subroutine.
that's the same as
NAME(@_);
right?

But I don't understand the difference between
NAME(LIST); # & is optional with parentheses.
and
&NAME(LIST); # Circumvent prototypes.

could you clarify that for me?
 
B

Ben Morrow

Well, now I looked in perlsub.
Quoting:
"
NAME(LIST); # & is optional with parentheses.
NAME LIST; # Parentheses optional if predeclared/imported.
&NAME(LIST); # Circumvent prototypes.
&NAME; # Makes current @_ visible to called subroutine.
"
and:
"A subroutine may be called using an explicit & prefix. The & is
optional in modern Perl..."

The second quote seems to contradict the first partly.

Yesss.... the first is right, though. The two forms have different
semantics.
The first I dont understand fully.
I think I understand
&NAME; # Makes current @_ visible to called subroutine.
that's the same as
NAME(@_);
right?

No. Firstly, it would be the same as &NAME(@_): prototypes are still ignored.
Secondly, consider

perl -le'sub a { &b(@_); print @_; &b; print @_ }
sub b { push @_, "b" }
a("a")'
a
ab

.. The first call copies the current @_ into b's @_; the second actually
gives b the *same* @_ as a had, so modifying it will modify a's. This
sort of strange-action-at-a-distance effect is very bad for
maintainability.
But I don't understand the difference between
NAME(LIST); # & is optional with parentheses.
and
&NAME(LIST); # Circumvent prototypes.

could you clarify that for me?

This is because you don't understand prototypes :). If you want to, read
the section on prototypes until you understand them; in the meanwhile,
suffice it to say that calling a sub with a prototype with & causes its
arguments to be parsed in a different way, and that the author of the
sub will not have been expecting this. Thus, anything might happen: you
should be sure you know what the effects will be before you do it. It
says somewhere in perlsub that this effect is left in as a way of
allowing experts to cheat if they need to: so yes, TMTOWTDI, but they do
different things :).

Ben
 
R

Rex Gustavus Adolphus

Paul Lalli said:
Efficiency isn't the issue here. These two programs are doing something
different. The first one is declaring $foo's scope to be the whole file;
the second one's scope is only within the while loop.
Make sure you know what you're telling your script to do before you worry
about how efficient it is.

Well, I'm thinking of the efficiency.
Thanks for your concern, but I already know the different scopes for
$foo in the two examples, believe me, I would not try to use $foo
outside the while loop in my second example.

(I always "use strict" by the way.)

I asked, because I was recommended earlier in this thread
to put variables in the smallest possible scope,
and I do that in the sense, that I avoid using global variables,
but I tend to declare variables used in loops (while, for) before the
loop starts,
like in my first example (even if they are not used outside the loop)
and that's because I have this feeling that it's a little bit
inefficient
to do it like in my second example.

/RGA
 
R

Rex Gustavus Adolphus

Ben Morrow said:
Yesss.... the first is right, though. The two forms have different
semantics.

It seems to me, the second quote needs to be fixed til next version then.
I think I have had that "& is optional" in the back of my head somewhere
This is because you don't understand prototypes :). If you want to, read
the section on prototypes until you understand them; in the meanwhile,
suffice it to say that calling a sub with a prototype with & causes its
arguments to be parsed in a different way, and that the author of the
sub will not have been expecting this. Thus, anything might happen: you
should be sure you know what the effects will be before you do it. It
says somewhere in perlsub that this effect is left in as a way of
allowing experts to cheat if they need to: so yes, TMTOWTDI, but they do
different things :).

Ben

Ben, I think I'll stop using &sub()-calls now.
I think it looks rather ugly anyway...

Thanks for your patience!
/RGA
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top