File::Slurp/IO::String/wantarray interaction bug

Discussion in 'Perl Misc' started by kj, Jun 11, 2010.

  1. kj

    kj Guest

    Consider the following demo script:

    # first line
    # second line

    use File::Slurp 'slurp';
    use IO::String;

    {
    my $string = slurp( $0 );
    my $io_string = IO::String->new( $string );
    printf ">>>%s<<<\n", scalar $io_string->READLINE;
    printf ">>>%s<<<\n", scalar $io_string->READLINE;
    }

    {
    my $io_string = IO::String->new( slurp( $0 ) );
    printf ">>>%s<<<\n", scalar $io_string->READLINE;
    printf ">>>%s<<<\n", scalar $io_string->READLINE;
    }

    __END__


    The only difference between the two blocks is that the first one
    assigns the value returned by slurp( $0 ) to the intermediate
    lexical variable $io_string, while the second one uses this value
    directly.

    As far as I can tell, the output of both blocks should be identical,
    but they are not even close. The output I get for the script above
    is:

    >>># first line

    <<<
    >>># second line

    <<<


    >>># first line

    <<<
    >>><<<



    Note that in the second block, the second line never gets printed;
    it is treated as an empty string.

    Stepping through the code with the debugger, I narrowed down the
    problem to the first line in the definition of IO::String::READLINE:

    sub READLINE
    {
    goto &getlines if wantarray;
    goto &getline;
    }

    Somehow, in the failing cases, the wantarray in the first line of
    READLINE evaluates to true, even though the original calling
    environment clearly specifies the scalar keyword. Therefore getlines
    gets inappropriately called, rather than the correct getline.

    How can the *really* force a scalar environment? (Clearly, the
    scalar keyword is not doing the job here.)

    FWIW, I'm using perl v5.10.0 on Ubuntu Linux.

    TIA!

    ~K
    kj, Jun 11, 2010
    #1
    1. Advertising

  2. kj

    kj Guest

    In <huuf1m$9kt$> kj <> writes:

    >Consider the following demo script:


    ># first line
    ># second line


    >use File::Slurp 'slurp';
    >use IO::String;


    >{
    > my $string = slurp( $0 );
    > my $io_string = IO::String->new( $string );
    > printf ">>>%s<<<\n", scalar $io_string->READLINE;
    > printf ">>>%s<<<\n", scalar $io_string->READLINE;
    >}


    >{
    > my $io_string = IO::String->new( slurp( $0 ) );
    > printf ">>>%s<<<\n", scalar $io_string->READLINE;
    > printf ">>>%s<<<\n", scalar $io_string->READLINE;
    >}


    >__END__



    >The only difference between the two blocks is that the first one
    >assigns the value returned by slurp( $0 ) to the intermediate
    >lexical variable $io_string, while the second one uses this value
    >directly.


    >As far as I can tell, the output of both blocks should be identical,
    >but they are not even close. The output I get for the script above
    >is:


    >>>># first line

    ><<<
    >>>># second line

    ><<<



    >>>># first line

    ><<<
    >>>><<<



    >Note that in the second block, the second line never gets printed;
    >it is treated as an empty string.


    >Stepping through the code with the debugger, I narrowed down the
    >problem to the first line in the definition of IO::String::READLINE:


    >sub READLINE
    >{
    > goto &getlines if wantarray;
    > goto &getline;
    >}


    >Somehow, in the failing cases, the wantarray in the first line of
    >READLINE evaluates to true, even though the original calling
    >environment clearly specifies the scalar keyword. Therefore getlines
    >gets inappropriately called, rather than the correct getline.


    >How can the *really* force a scalar environment? (Clearly, the
    >scalar keyword is not doing the job here.)



    OK, I found out one more detail. If I replace the original second block

    {
    my $io_string = IO::String->new( slurp( $0 ) );
    printf ">>>%s<<<\n", scalar $io_string->READLINE;
    printf ">>>%s<<<\n", scalar $io_string->READLINE;
    }

    with

    {
    my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed
    printf ">>>%s<<<\n", scalar $io_string->READLINE;
    printf ">>>%s<<<\n", scalar $io_string->READLINE;
    }

    now the output is identical for both blocks.

    Still, this is quite a curveball. I see that in the original
    version, the call to slurp was happenning in a list context, and
    this probably messes up the string that actually gets used to define
    the IO::String object. But still, this does not explain why READLINE
    believes its in a list context. (I'm sorry that I can't show this
    easily; one needs to step there with the debugger).

    ~K
    kj, Jun 12, 2010
    #2
    1. Advertising

  3. kj

    Uri Guttman Guest

    >>>>> "k" == kj <> writes:

    k> In <huuf1m$9kt$> kj <> writes:
    >> Consider the following demo script:


    >> # first line
    >> # second line


    >> use File::Slurp 'slurp';


    most people use the read_file sub. slurp is just an alias to it for
    backwards compatability.

    >> use IO::String;


    why do you play with io::string? perl's open can do that builtin these
    days. it does require a scalar var and can't do it on data but that
    isn't much of an issue.

    and a general question, what are you trying to do here??

    >> {
    >> my $string = slurp( $0 );
    >> my $io_string = IO::String->new( $string );
    >> printf ">>>%s<<<\n", scalar $io_string->READLINE;
    >> printf ">>>%s<<<\n", scalar $io_string->READLINE;
    >> }


    >> {
    >> my $io_string = IO::String->new( slurp( $0 ) );


    all sub/method calls pass list context to their arguments. remember,
    those args get set into the @_ array so that makes sense. otherwise you
    couldn't pass in an array and it would be converted to its count which
    is usually not wanted in @_.

    >> printf ">>>%s<<<\n", scalar $io_string->READLINE;
    >> printf ">>>%s<<<\n", scalar $io_string->READLINE;
    >> }


    printf also like other funcs which can take a list of args, has list
    context for everything past the format arg.

    k> OK, I found out one more detail. If I replace the original second block

    k> {
    k> my $io_string = IO::String->new( slurp( $0 ) );
    k> printf ">>>%s<<<\n", scalar $io_string->READLINE;
    k> printf ">>>%s<<<\n", scalar $io_string->READLINE;
    k> }

    k> with

    k> {
    k> my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed

    well, now it slurps the file to a scalar and not a list of lines.

    and the IO::Scalar docs disagree with your call:

    new [ARGS...]
    Class method. Return a new, unattached scalar handle. If any
    arguments are given, they're sent to open().

    open [SCALARREF]
    Instance method. Open the scalar handle on a new scalar, pointed
    to by SCALARREF. If no SCALARREF is given, a "private" scalar is
    created to hold the file data.

    Returns the self object on success, undefined on error.

    so you aren't even calling it correctly. you need to pass it a scalar
    ref and you are passing it a scalar value or a scalar. maybe it can
    handle that but it doesn't say that in the docs (at least what i read).

    k> Still, this is quite a curveball. I see that in the original
    k> version, the call to slurp was happenning in a list context, and
    k> this probably messes up the string that actually gets used to define
    k> the IO::String object. But still, this does not explain why READLINE
    k> believes its in a list context. (I'm sorry that I can't show this
    k> easily; one needs to step there with the debugger).

    as i said, see the docs for printf. after the format it expects a list
    so that call will be in list context.

    uri

    --
    Uri Guttman ------ -------- http://www.sysarch.com --
    ----- Perl Code Review , Architecture, Development, Training, Support ------
    --------- Gourmet Hot Cocoa Mix ---- http://bestfriendscocoa.com ---------
    Uri Guttman, Jun 12, 2010
    #3
  4. kj

    Willem Guest

    kj wrote:
    ) OK, I found out one more detail. If I replace the original second block
    )
    ) {
    ) my $io_string = IO::String->new( slurp( $0 ) );
    ) printf ">>>%s<<<\n", scalar $io_string->READLINE;
    ) printf ">>>%s<<<\n", scalar $io_string->READLINE;
    ) }
    )
    ) with
    )
    ) {
    ) my $io_string = IO::String->new( scalar slurp( $0 ) ); # <- only this line changed
    ) printf ">>>%s<<<\n", scalar $io_string->READLINE;
    ) printf ">>>%s<<<\n", scalar $io_string->READLINE;
    ) }
    )
    ) now the output is identical for both blocks.
    )
    ) Still, this is quite a curveball. I see that in the original
    ) version, the call to slurp was happenning in a list context, and
    ) this probably messes up the string that actually gets used to define
    ) the IO::String object. But still, this does not explain why READLINE
    ) believes its in a list context. (I'm sorry that I can't show this
    ) easily; one needs to step there with the debugger).

    When I saw your first post, it was immediately apparent to me that the
    call to slurp was being evaluated in list context, and that caused it
    to pass only the first line to the IO::String object.

    The READLINE context is a red herring. Maybe you misinterpreted the
    debugger output, maybe the debugger did something strange, maybe something
    else happened. In any case, it's quite clear that it *is* being called in
    scalar context, because that final fix works like it should.


    SaSW, Willem
    --
    Disclaimer: I am in no way responsible for any of the statements
    made in the above text. For all I know I might be
    drugged or something..
    No I'm not paranoid. You all think I'm paranoid, don't you !
    #EOT
    Willem, Jun 12, 2010
    #4
  5. kj

    kj Guest

    In <> Willem <> writes:

    >...Maybe you misinterpreted the
    >debugger output, maybe the debugger did something strange, maybe something
    >else happened.


    Well, it looks that it's one of those. To determine the calling
    context I was halting the execution within the READLINE method,
    and then printing the value of wantarray from the debugger prompt
    like this:

    DB<1> p wantarray
    1

    Apparentely, in the debugger "p wantarray" always produces this
    result, irrespective of the value of wantarray in the executing
    program. (That's quite the "banana peel", IMHO.)

    Poking around in the perldebug man page I discovered that the right
    way to determine the calling context from within the debugger is
    to use the T command:

    DB<4> T
    $ = IO::String::READLINE(ref(IO::String)) called from file `t/wantarraybug.pl' line 28

    The leading '$' is shorthand for "called in scalar context".

    Thanks for your post!

    ~K
    kj, Jun 14, 2010
    #5
  6. kj

    kj Guest

    In <> "Uri Guttman" <> writes:

    >and the IO::Scalar docs disagree with your call:


    > new [ARGS...]
    > Class method. Return a new, unattached scalar handle. If any
    > arguments are given, they're sent to open().


    > open [SCALARREF]
    > Instance method. Open the scalar handle on a new scalar, pointed
    > to by SCALARREF. If no SCALARREF is given, a "private" scalar is
    > created to hold the file data.


    > Returns the self object on success, undefined on error.


    >so you aren't even calling it correctly.


    I'm using IO::String, not IO::Scalar. My IO::String docs says:

    $io = IO::String->new
    $io = IO::String->new( $string )
    The constructor returns a newly-created "IO::String"
    object. It takes an optional argument, which is the
    string to read from or write into. ...

    As I described in the post before this one, what led me down the
    wrong path was how I was determining the value of wantarray within
    the debugger.

    Thanks for your comments!

    ~K
    kj, Jun 14, 2010
    #6
  7. On 2010-06-14, kj <> wrote:
    > Well, it looks that it's one of those. To determine the calling
    > context I was halting the execution within the READLINE method,
    > and then printing the value of wantarray from the debugger prompt
    > like this:
    >
    > DB<1> p wantarray
    > 1
    >
    > Apparentely, in the debugger "p wantarray" always produces this
    > result, irrespective of the value of wantarray in the executing
    > program.


    Sure, how else would it be? `p' evaluates its expression in list context...

    Yours,
    Ilya
    Ilya Zakharevich, Jun 15, 2010
    #7
  8. kj

    kj Guest

    In <> Ilya Zakharevich <> writes:

    >On 2010-06-14, kj <> wrote:
    >> Well, it looks that it's one of those. To determine the calling
    >> context I was halting the execution within the READLINE method,
    >> and then printing the value of wantarray from the debugger prompt
    >> like this:
    >>
    >> DB<1> p wantarray
    >> 1
    >>
    >> Apparentely, in the debugger "p wantarray" always produces this
    >> result, irrespective of the value of wantarray in the executing
    >> program.


    >Sure, how else would it be? `p' evaluates its expression in list context...


    Yes, in retrospect I see that what I was doing is pretty dumb.
    kj, Jun 15, 2010
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dick Davies
    Replies:
    1
    Views:
    114
    Gavin Sinclair
    Sep 29, 2005
  2. Wes Gamble
    Replies:
    7
    Views:
    131
    Lyle Johnson
    Mar 23, 2006
  3. Charles R. Thompson
    Replies:
    6
    Views:
    152
    Ben Liddicott
    Jan 13, 2004
  4. Stumproot

    if(wantarray){...}else{...}

    Stumproot, Apr 10, 2005, in forum: Perl Misc
    Replies:
    13
    Views:
    171
    A. Sinan Unur
    Apr 11, 2005
  5. Ronald Fischer

    Trouble in wantarray context

    Ronald Fischer, Nov 21, 2005, in forum: Perl Misc
    Replies:
    4
    Views:
    104
    Ronald Fischer
    Nov 28, 2005
Loading...

Share This Page