File::Slurp/IO::String/wantarray interaction bug

[kj]

Consider the following demo script:

# first line
# second line

use File::Slurp 'slurp';
use IO::String;

{
my $string = slurp( $0 );
my $io_string = IO::String->new( $string );
printf ">>>%s<<<\n", scalar $io_string->READLINE;
printf ">>>%s<<<\n", scalar $io_string->READLINE;
}

{
my $io_string = IO::String->new( slurp( $0 ) );
printf ">>>%s<<<\n", scalar $io_string->READLINE;
printf ">>>%s<<<\n", scalar $io_string->READLINE;
}

__END__


The only difference between the two blocks is that the first one
assigns the value returned by slurp( $0 ) to the intermediate
lexical variable $io_string, while the second one uses this value
directly.

As far as I can tell, the output of both blocks should be identical,
but they are not even close. The output I get for the script above
is:

<<<

Note that in the second block, the second line never gets printed;
it is treated as an empty string.

Stepping through the code with the debugger, I narrowed down the
problem to the first line in the definition of IO::String::READLINE:

sub READLINE
{
goto &getlines if wantarray;
goto &getline;
}

Somehow, in the failing cases, the wantarray in the first line of
READLINE evaluates to true, even though the original calling
environment clearly specifies the scalar keyword. Therefore getlines
gets inappropriately called, rather than the correct getline.

How can the *really* force a scalar environment? (Clearly, the
scalar keyword is not doing the job here.)

FWIW, I'm using perl v5.10.0 on Ubuntu Linux.

TIA!

~K

 

[kj]

Consider the following demo script:

# first line
# second line

use File::Slurp 'slurp';
use IO::String;

{
my $string = slurp( $0 );
my $io_string = IO::String->new( $string );
printf ">>>%s<<<\n", scalar $io_string->READLINE;
printf ">>>%s<<<\n", scalar $io_string->READLINE;
}

{
my $io_string = IO::String->new( slurp( $0 ) );
printf ">>>%s<<<\n", scalar $io_string->READLINE;
printf ">>>%s<<<\n", scalar $io_string->READLINE;
}


The only difference between the two blocks is that the first one
assigns the value returned by slurp( $0 ) to the intermediate
lexical variable $io_string, while the second one uses this value
directly.

As far as I can tell, the output of both blocks should be identical,
but they are not even close. The output I get for the script above
is:



Note that in the second block, the second line never gets printed;
it is treated as an empty string.

Stepping through the code with the debugger, I narrowed down the
problem to the first line in the definition of IO::String::READLINE:

sub READLINE
{
goto &getlines if wantarray;
goto &getline;
}

Somehow, in the failing cases, the wantarray in the first line of
READLINE evaluates to true, even though the original calling
environment clearly specifies the scalar keyword. Therefore getlines
gets inappropriately called, rather than the correct getline.

How can the *really* force a scalar environment? (Clearly, the
scalar keyword is not doing the job here.)

OK, I found out one more detail. If I replace the original second block

{
my $io_string = IO::String->new( slurp( $0 ) );
printf ">>>%s<<<\n", scalar $io_string->READLINE;
printf ">>>%s<<<\n", scalar $io_string->READLINE;
}

with

{
my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed
printf ">>>%s<<<\n", scalar $io_string->READLINE;
printf ">>>%s<<<\n", scalar $io_string->READLINE;
}

now the output is identical for both blocks.

Still, this is quite a curveball. I see that in the original
version, the call to slurp was happenning in a list context, and
this probably messes up the string that actually gets used to define
the IO::String object. But still, this does not explain why READLINE
believes its in a list context. (I'm sorry that I can't show this
easily; one needs to step there with the debugger).

~K

 

[Uri Guttman]

most people use the read_file sub. slurp is just an alias to it for
backwards compatability.

why do you play with io::string? perl's open can do that builtin these
days. it does require a scalar var and can't do it on data but that
isn't much of an issue.

and a general question, what are you trying to do here??

all sub/method calls pass list context to their arguments. remember,
those args get set into the @_ array so that makes sense. otherwise you
couldn't pass in an array and it would be converted to its count which
is usually not wanted in @_.

printf also like other funcs which can take a list of args, has list
context for everything past the format arg.

k> OK, I found out one more detail. If I replace the original second block

k> {
k> my $io_string = IO::String->new( slurp( $0 ) );
k> printf ">>>%s<<<\n", scalar $io_string->READLINE;
k> printf ">>>%s<<<\n", scalar $io_string->READLINE;
k> }

k> with

k> {
k> my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed

well, now it slurps the file to a scalar and not a list of lines.

and the IO::Scalar docs disagree with your call:

new [ARGS...]
Class method. Return a new, unattached scalar handle. If any
arguments are given, they're sent to open().

open [SCALARREF]
Instance method. Open the scalar handle on a new scalar, pointed
to by SCALARREF. If no SCALARREF is given, a "private" scalar is
created to hold the file data.

Returns the self object on success, undefined on error.

so you aren't even calling it correctly. you need to pass it a scalar
ref and you are passing it a scalar value or a scalar. maybe it can
handle that but it doesn't say that in the docs (at least what i read).

k> Still, this is quite a curveball. I see that in the original
k> version, the call to slurp was happenning in a list context, and
k> this probably messes up the string that actually gets used to define
k> the IO::String object. But still, this does not explain why READLINE
k> believes its in a list context. (I'm sorry that I can't show this
k> easily; one needs to step there with the debugger).

as i said, see the docs for printf. after the format it expects a list
so that call will be in list context.

uri

 

[Willem]

kj wrote:
) OK, I found out one more detail. If I replace the original second block
)
) {
) my $io_string = IO::String->new( slurp( $0 ) );
) printf ">>>%s<<<\n", scalar $io_string->READLINE;
) printf ">>>%s<<<\n", scalar $io_string->READLINE;
) }
)
) with
)
) {
) my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed
) printf ">>>%s<<<\n", scalar $io_string->READLINE;
) printf ">>>%s<<<\n", scalar $io_string->READLINE;
) }
)
) now the output is identical for both blocks.
)
) Still, this is quite a curveball. I see that in the original
) version, the call to slurp was happenning in a list context, and
) this probably messes up the string that actually gets used to define
) the IO::String object. But still, this does not explain why READLINE
) believes its in a list context. (I'm sorry that I can't show this
) easily; one needs to step there with the debugger).

When I saw your first post, it was immediately apparent to me that the
call to slurp was being evaluated in list context, and that caused it
to pass only the first line to the IO::String object.

The READLINE context is a red herring. Maybe you misinterpreted the
debugger output, maybe the debugger did something strange, maybe something
else happened. In any case, it's quite clear that it *is* being called in
scalar context, because that final fix works like it should.


SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT

 

[kj]

...Maybe you misinterpreted the
debugger output, maybe the debugger did something strange, maybe something
else happened.

Well, it looks that it's one of those. To determine the calling
context I was halting the execution within the READLINE method,
and then printing the value of wantarray from the debugger prompt
like this:

DB<1> p wantarray
1

Apparentely, in the debugger "p wantarray" always produces this
result, irrespective of the value of wantarray in the executing
program. (That's quite the "banana peel", IMHO.)

Poking around in the perldebug man page I discovered that the right
way to determine the calling context from within the debugger is
to use the T command:

DB<4> T
$ = IO::String::READLINE(ref(IO::String)) called from file `t/wantarraybug.pl' line 28

The leading '$' is shorthand for "called in scalar context".

Thanks for your post!

~K

 

[kj]

and the IO::Scalar docs disagree with your call:

new [ARGS...]
Class method. Return a new, unattached scalar handle. If any
arguments are given, they're sent to open().

open [SCALARREF]
Instance method. Open the scalar handle on a new scalar, pointed
to by SCALARREF. If no SCALARREF is given, a "private" scalar is
created to hold the file data.

Returns the self object on success, undefined on error.

so you aren't even calling it correctly.

I'm using IO::String, not IO::Scalar. My IO::String docs says:

$io = IO::String->new
$io = IO::String->new( $string )
The constructor returns a newly-created "IO::String"
object. It takes an optional argument, which is the
string to read from or write into. ...

As I described in the post before this one, what led me down the
wrong path was how I was determining the value of wantarray within
the debugger.

Thanks for your comments!

~K

 

[Ilya Zakharevich]

Well, it looks that it's one of those. To determine the calling
context I was halting the execution within the READLINE method,
and then printing the value of wantarray from the debugger prompt
like this:

DB<1> p wantarray
1

Apparentely, in the debugger "p wantarray" always produces this
result, irrespective of the value of wantarray in the executing
program.

Sure, how else would it be? `p' evaluates its expression in list context...

Yours,
Ilya

 

[kj]

Sure, how else would it be? `p' evaluates its expression in list context...

Yes, in retrospect I see that what I was doing is pretty dumb.