binmode blues

Discussion in 'Perl Misc' started by jidanni@jidanni.org, Sep 11, 2008.

  1. Guest

    What is wrong with
    $ cat normalize
    #!/usr/bin/perl
    binmode STDIN, ":utf8"; binmode STDOUT, ":utf8";
    use Unicode::Normalize q(decompose);
    local $/; $_=<>; print decompose($_);

    which is causing these to differ?:
    $ normalize 4.htm |head -c 11|od -x|head -n 1
    0000000 2020 cc61 c282 c280 20a2 0061
    $ normalize < 4.htm |head -c 11|od -x|head -n 1
    0000000 2020 80e2 20a2 b0e5 e88e 00a6

    I.e., why must I remember to use a "<" to get proper results?
     
    , Sep 11, 2008
    #1
    1. Advertising

  2. brian d foy Guest

    In article <>, <> wrote:


    > which is causing these to differ?:
    > $ normalize 4.htm |head -c 11|od -x|head -n 1
    > 0000000 2020 cc61 c282 c280 20a2 0061


    This one is reading from ARGV. What happens when you bin mode that
    filehandle too?

    > $ normalize < 4.htm |head -c 11|od -x|head -n 1
    > 0000000 2020 80e2 20a2 b0e5 e88e 00a6


    This one is reading from STDIN.
     
    brian d foy, Sep 11, 2008
    #2
    1. Advertising

  3. wrote:
    > What is wrong with
    > $ cat normalize
    > #!/usr/bin/perl
    > binmode STDIN, ":utf8"; binmode STDOUT, ":utf8";
    > use Unicode::Normalize q(decompose);
    > local $/; $_=<>; print decompose($_);


    > which is causing these to differ?:
    > $ normalize 4.htm |head -c 11|od -x|head -n 1
    > 0000000 2020 cc61 c282 c280 20a2 0061
    > $ normalize < 4.htm |head -c 11|od -x|head -n 1
    > 0000000 2020 80e2 20a2 b0e5 e88e 00a6


    > I.e., why must I remember to use a "<" to get proper results?


    23:03:38 38 [0:0]$ perl -wle '
    $x = <>; system qq|ls -l /proc/$$/fd|' /etc/passwd
    Name "main::x" used only once: possible typo at -e line 1.
    total 0
    lrwx------ 1 whynot whynot 64 2008-09-11 23:31 0 -> /dev/pts/1
    lrwx------ 1 whynot whynot 64 2008-09-11 23:31 1 -> /dev/pts/1
    lrwx------ 1 whynot whynot 64 2008-09-11 23:31 2 -> /dev/pts/1
    lr-x------ 1 whynot whynot 64 2008-09-11 23:31 3 -> /etc/passwd
    ^^^^^^^^^^^^^^^^
    lr-x------ 1 whynot whynot 64 2008-09-11 23:31 4 -> pipe:[4517507]
    l-wx------ 1 whynot whynot 64 2008-09-11 23:31 5 -> pipe:[4517507]
    23:31:10 39 [0:0]$ perl -wle '
    $x = <>; system qq|ls -l /proc/$$/fd|' </etc/passwd
    Name "main::x" used only once: possible typo at -e line 1.
    total 0
    lr-x------ 1 whynot whynot 64 2008-09-11 23:31 0 -> /etc/passwd
    ^^^^^^^^^^^^^^^^
    lrwx------ 1 whynot whynot 64 2008-09-11 23:31 1 -> /dev/pts/1
    lrwx------ 1 whynot whynot 64 2008-09-11 23:31 2 -> /dev/pts/1
    lr-x------ 1 whynot whynot 64 2008-09-11 23:31 3 -> pipe:[4517541]


    --
    Torvalds' goal for Linux is very simple: World Domination
     
    Eric Pozharski, Sep 11, 2008
    #3
  4. Ben Morrow Guest

    Quoth :
    > bdf> This one is reading from ARGV. What happens when you bin mode that
    > bdf> filehandle too?
    >
    > I see. OK, so what should I write instead of just
    > > binmode STDIN, ":utf8"; binmode STDOUT, ":utf8";

    > so that it will work no matter if my program is called with any of
    > $ myprogram < file1
    > $ myprogram file1
    > $ myprogram file1 file2 file3 ...
    > $ myprogram file1 file2 file3 < file4
    > Ah, for(STDIN,STDOUT,@ARGV){binmode $_, ":utf8"}... shot down real fast:
    > Bareword "STDIN" not allowed while "strict subs"... as we see, yes I don't
    > know what I'm doing.


    Global filehandles are a little odd in Perl. You want

    for (\*STDIN, \*STDOUT, \*ARGV) {
    binmode $_, ":utf8";
    }

    except you *really* don't want to be using the :utf8 layer. Use
    :encoding(utf8) instead: it's much safer in te face of invalid data.

    Ben

    --
    Musica Dei donum optimi, trahit homines, trahit deos. |
    Musica truces mollit animos, tristesque mentes erigit. |
    Musica vel ipsas arbores et horridas movet feras. |
     
    Ben Morrow, Sep 12, 2008
    #4
  5. Ben Morrow wrote:
    > Quoth :
    >> bdf> This one is reading from ARGV. What happens when you bin mode that
    >> bdf> filehandle too?
    >>
    >> I see. OK, so what should I write instead of just
    >>> binmode STDIN, ":utf8"; binmode STDOUT, ":utf8";

    >> so that it will work no matter if my program is called with any of
    >> $ myprogram < file1
    >> $ myprogram file1
    >> $ myprogram file1 file2 file3 ...
    >> $ myprogram file1 file2 file3 < file4
    >> Ah, for(STDIN,STDOUT,@ARGV){binmode $_, ":utf8"}... shot down real fast:
    >> Bareword "STDIN" not allowed while "strict subs"... as we see, yes I don't
    >> know what I'm doing.

    >
    > Global filehandles are a little odd in Perl. You want
    >
    > for (\*STDIN, \*STDOUT, \*ARGV) {
    > binmode $_, ":utf8";
    > }


    And you may want to also include ARGVOUT in there? Of course you
    *could* do it like this instead:

    use open qw/:std :utf8/;


    > except you *really* don't want to be using the :utf8 layer. Use
    > :encoding(utf8) instead: it's much safer in te face of invalid data.


    use open qw/:std :encoding(utf8)/;



    John
    --
    Perl isn't a toolbox, but a small machine shop where you
    can special-order certain sorts of tools at low cost and
    in short order. -- Larry Wall
     
    John W. Krahn, Sep 12, 2008
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dmitry Borodaenko

    StringIO#binmode: bug in cgi.rb and yaml.rb

    Dmitry Borodaenko, Aug 20, 2003, in forum: Ruby
    Replies:
    2
    Views:
    187
    Nobuyoshi Nakada
    Aug 23, 2003
  2. Henning Koch

    pstore.rb not operating in binmode

    Henning Koch, Oct 4, 2004, in forum: Ruby
    Replies:
    4
    Views:
    135
    Henrik Horneber
    Oct 5, 2004
  3. J. Romano

    binmode and the diamond operator

    J. Romano, Nov 8, 2003, in forum: Perl Misc
    Replies:
    6
    Views:
    199
    J. Romano
    Nov 17, 2003
  4. Alain Star
    Replies:
    1
    Views:
    281
    Brian McCauley
    Dec 13, 2004
  5. dearcilla

    binmode clobbers :via

    dearcilla, Aug 1, 2005, in forum: Perl Misc
    Replies:
    1
    Views:
    99
Loading...

Share This Page