Unintuitive expression evaluation

Discussion in 'Perl Misc' started by Bill Davidsen, Oct 27, 2005.

  1. I am reading a log file which has 32 bit unsigned values, and need to
    take the difference between them. If the difference is negative I assume
    that the data counter has rolled over, and I want to add 231 to get the
    correct value. For readability I wanted to put the value in a variable
    so it would be obvious what was happening.

    So I wrote:
    $Roll32 = (1 << 32); # the way I would for a C macro
    but the value was (after I did some looking) one! Then I wrote:
    $Roll32 = 1 << 32; # in case the parens were a issue
    bit it was still one, so I wrote:
    $Roll32 = ( 4 * (1 << 30) ); # which works (4G)

    This is the first time perl has failed to do int=>double conversion when
    expected (by me). Just a note in case someone else is ever doing
    similar, perl on Linux 32 bit, v5.6.1, v5.8.0, v5.8.5 builds.

    --
    bill davidsen
    SBC/Prodigy Yorktown Heights NY data center
    http://newsgroups.news.prodigy.com
     
    Bill Davidsen, Oct 27, 2005
    #1
    1. Advertising

  2. Bill Davidsen <> wrote in
    news:mNa8f.3023$:

    > I am reading a log file which has 32 bit unsigned values, and need to
    > take the difference between them. If the difference is negative I
    > assume that the data counter has rolled over, and I want to add 231 to
    > get the correct value. For readability I wanted to put the value in a
    > variable so it would be obvious what was happening.
    >
    > So I wrote:
    > $Roll32 = (1 << 32); # the way I would for a C macro
    > but the value was (after I did some looking) one! Then I wrote:
    > $Roll32 = 1 << 32; # in case the parens were a issue
    > bit it was still one, so I wrote:
    > $Roll32 = ( 4 * (1 << 30) ); # which works (4G)
    >
    > This is the first time perl has failed to do int=>double conversion
    > when expected (by me).


    The only thing that failed in this case is you.

    > Just a note in case someone else is ever doing
    > similar, perl on Linux 32 bit, v5.6.1, v5.8.0, v5.8.5 builds.


    You mean "just in case someone else refuses to read the documentation"?

    perldoc perlop:

    Shift Operators
    ....
    Either way, the implementation isn't going to generate results larger
    than the size of the integer type Perl was built with (32 bits or 64
    bits).

    The result of overflowing the range of the integers is undefined
    because it is undefined also in C. In other words, using 32-bit
    integers, "1 << 32" is undefined. Shifting by a negative number
    of bits is also undefined.

    How hard is it to read this, and understand it?

    Sinan

    --
    A. Sinan Unur <>
    (reverse each component andremove .invalid for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
     
    A. Sinan Unur, Oct 27, 2005
    #2
    1. Advertising

  3. Also sprach A. Sinan Unur:
    > Bill Davidsen <> wrote in
    > news:mNa8f.3023$:
    >
    >> I am reading a log file which has 32 bit unsigned values, and need to
    >> take the difference between them. If the difference is negative I
    >> assume that the data counter has rolled over, and I want to add 231 to
    >> get the correct value. For readability I wanted to put the value in a
    >> variable so it would be obvious what was happening.
    >>
    >> So I wrote:
    >> $Roll32 = (1 << 32); # the way I would for a C macro
    >> but the value was (after I did some looking) one! Then I wrote:
    >> $Roll32 = 1 << 32; # in case the parens were a issue
    >> bit it was still one, so I wrote:
    >> $Roll32 = ( 4 * (1 << 30) ); # which works (4G)
    >>
    >> This is the first time perl has failed to do int=>double conversion
    >> when expected (by me).

    >
    > The only thing that failed in this case is you.
    >
    >> Just a note in case someone else is ever doing
    >> similar, perl on Linux 32 bit, v5.6.1, v5.8.0, v5.8.5 builds.

    >
    > You mean "just in case someone else refuses to read the documentation"?
    >
    > perldoc perlop:
    >
    > Shift Operators
    > ...
    > Either way, the implementation isn't going to generate results larger
    > than the size of the integer type Perl was built with (32 bits or 64
    > bits).
    >
    > The result of overflowing the range of the integers is undefined
    > because it is undefined also in C. In other words, using 32-bit
    > integers, "1 << 32" is undefined. Shifting by a negative number
    > of bits is also undefined.
    >
    > How hard is it to read this, and understand it?


    Considering that

    $Roll32 = ( 4 * (1 << 30) );

    gives the expected result, these paragraphs may be a tad difficult to
    understand, particularly for people with a C-background.

    Replacing a two-bit shift with a multiplication by four works because
    perl internally upgrades (as needed) the scalar from holding a plain
    integer to holding a double when doing the multiplication.

    I think the left-shift operator could be made smarter so that it always
    works up to 2**64 - 1, even on 32bit machines.

    Tassilo
    --
    use bigint;
    $n=71423350343770280161397026330337371139054411854220053437565440;
    $m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
     
    Tassilo v. Parseval, Oct 28, 2005
    #3
  4. Bill Davidsen

    Anno Siegel Guest

    Tassilo v. Parseval <> wrote in comp.lang.perl.misc:
    > Also sprach A. Sinan Unur:
    > > Bill Davidsen <> wrote in
    > > news:mNa8f.3023$:


    > I think the left-shift operator could be made smarter so that it always
    > works up to 2**64 - 1, even on 32bit machines.


    ....but only if the bit pattern can be preserved. It wouldn't do to return
    the numeric equivalent as a float. So there must be support for 64 bit
    integers, either in hardware or in software. I don't think Perl supplies
    that if the native C doesn't.

    Anno
    --
    If you want to post a followup via groups.google.com, don't use
    the broken "Reply" link at the bottom of the article. Click on
    "show options" at the top of the article, then click on the
    "Reply" at the bottom of the article headers.
     
    Anno Siegel, Oct 28, 2005
    #4
  5. Bill Davidsen

    Dr.Ruud Guest

    Bill Davidsen schreef:

    > (1 << 32)


    That is a 33-bit value.

    --
    Affijn, Ruud

    "Gewoon is een tijger."
     
    Dr.Ruud, Oct 28, 2005
    #5
  6. Also sprach Anno Siegel:

    > Tassilo v. Parseval <> wrote in comp.lang.perl.misc:
    >> Also sprach A. Sinan Unur:
    >> > Bill Davidsen <> wrote in
    >> > news:mNa8f.3023$:

    >
    >> I think the left-shift operator could be made smarter so that it always
    >> works up to 2**64 - 1, even on 32bit machines.

    >
    > ...but only if the bit pattern can be preserved. It wouldn't do to return
    > the numeric equivalent as a float. So there must be support for 64 bit
    > integers, either in hardware or in software. I don't think Perl supplies
    > that if the native C doesn't.


    Yes, the assumption was that some sort of 64bit integer exists. If it
    doesn't, left-shift will remain as it is now. The patch that should do
    that is:

    --- perl-p-5.8.0@25859~/pp.c 2005-10-27 08:58:48.000000000 +0200
    +++ perl-p-5.8.0@25859/pp.c 2005-10-28 08:42:45.000000000 +0200
    @@ -1627,12 +1627,32 @@ PP(pp_left_shift)
    {
    const IV shift = POPi;
    if (PL_op->op_private & HINT_INTEGER) {
    +#if !defined(USE_64_BIT_INT) && defined(I64TYPE)
    + I64TYPE i = TOPi;
    + i <<= shift;
    + if (i > I32_MAX) {
    + SETn((NV)i);
    + SvNOK_only(TARG);
    + } else
    + SETi(i);
    +#else
    IV i = TOPi;
    SETi(i << shift);
    +#endif
    }
    else {
    +#if !defined(USE_64_BIT_INT) && defined(U64TYPE)
    + U64TYPE u = TOPu;
    + u <<= shift;
    + if (u > U32_MAX) {
    + SETn((NV)u);
    + SvNOK_only(TARG);
    + } else
    + SETu(u);
    +#else
    UV u = TOPu;
    SETu(u << shift);
    +#endif
    }
    RETURN;
    }

    That fails two core-tests, though, and I suspect it's even
    backwards-incompatible because there were oudoubtedtly blockheads out
    there that assumed some sort of defined behaviour from overflowing
    left-shifts in their scripts.

    Nonetheless I passed that draft to the porters to see what they think
    about it.

    Tassilo
    --
    use bigint;
    $n=71423350343770280161397026330337371139054411854220053437565440;
    $m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
     
    Tassilo v. Parseval, Oct 28, 2005
    #6
  7. A. Sinan Unur wrote:
    > Bill Davidsen <> wrote in
    > news:mNa8f.3023$:
    >
    >
    >>I am reading a log file which has 32 bit unsigned values, and need to
    >>take the difference between them. If the difference is negative I
    >>assume that the data counter has rolled over, and I want to add 231 to
    >>get the correct value. For readability I wanted to put the value in a
    >>variable so it would be obvious what was happening.
    >>
    >>So I wrote:
    >> $Roll32 = (1 << 32); # the way I would for a C macro
    >>but the value was (after I did some looking) one! Then I wrote:
    >> $Roll32 = 1 << 32; # in case the parens were a issue
    >>bit it was still one, so I wrote:
    >> $Roll32 = ( 4 * (1 << 30) ); # which works (4G)
    >>
    >>This is the first time perl has failed to do int=>double conversion
    >>when expected (by me).

    >
    >
    > The only thing that failed in this case is you.


    Sorry, I didn't realize the word "unintuitive" would not be understood
    by some people.

    This works on 64 bit machines. This works on 32 bit machines if you
    compile for 64 bit integers. This doesn't work on 32 bit machines
    without 64 bit integers. However, it does work if you do it at runtime
    instead of compile time:

    oddball:davidsen> perl -e '
    @x=(4,1);
    for $a (@x) {
    $b=$a<<29;
    $c=$a<<30;
    print "$a $b $c\n"
    }'
    4 2147483648 0
    1 536870912 1073741824


    A feature which works in some places and not in others, and which
    doesn't generate compiler warnings in cases where it clearly isn't going
    to work certainly qualifies as "unintuitive" for me.

    --
    bill davidsen
    SBC/Prodigy Yorktown Heights NY data center
    http://newsgroups.news.prodigy.com
     
    Bill Davidsen, Oct 28, 2005
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Bob van der Poel

    unintuitive dict timings

    Bob van der Poel, Oct 12, 2003, in forum: Python
    Replies:
    5
    Views:
    359
    Dave Benjamin
    Oct 13, 2003
  2. Ilias Lazaridis
    Replies:
    2
    Views:
    410
    Ilias Lazaridis
    Apr 24, 2005
  3. Ilias Lazaridis
    Replies:
    74
    Views:
    804
    Ilias Lazaridis
    Apr 4, 2005
  4. John Woods
    Replies:
    10
    Views:
    187
    Joe Peck
    Dec 18, 2008
  5. Nick Brown
    Replies:
    19
    Views:
    223
    Michael Morin
    Aug 21, 2008
Loading...

Share This Page