strange calculation results...?

Discussion in 'Java' started by Jeremy, Feb 18, 2004.

  1. Jeremy

    Jeremy Guest

    hi,

    when performing some calculations for display in a table, I
    occasionally get the value '04294967295' which is equivalent to 2^32
    -1. This value is incorrect as the numbers I'm dealing with are not of
    this order.

    I know the above details are pretty sketchy, but this looks like some
    kind of overflow/underflow. Does anybody have any idea how this number
    could crop up?

    btw I'm using IBM jdk 1.4.0 on linux.
     
    Jeremy, Feb 18, 2004
    #1
    1. Advertising

  2. Jeremy

    Jim White Guest

    Jeremy wrote:

    > when performing some calculations for display in a table, I
    > occasionally get the value '04294967295' which is equivalent to 2^32
    > -1. This value is incorrect as the numbers I'm dealing with are not of
    > this order.
    >
    > I know the above details are pretty sketchy, but this looks like some
    > kind of overflow/underflow. Does anybody have any idea how this number
    > could crop up?


    Most likely is that you've taken a Java "int" (32 bit two's complement
    integer) -1 and converted it to an unsigned integer for display.
    Perhaps some code like: ((long) i) & 0xFFFFFFFFL, or something suitable
    in your display formatting code to show unsigned.

    Jim
     
    Jim White, Feb 18, 2004
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Monique Y. Mudama
    Replies:
    1
    Views:
    478
    Monique Y. Mudama
    Jun 28, 2005
  2. Badass Scotsman
    Replies:
    1
    Views:
    377
    S. Justin Gengo
    Apr 24, 2006
  3. Vedran

    Presenting calculation results

    Vedran, Sep 10, 2008, in forum: Python
    Replies:
    1
    Views:
    306
    Marco Bizzarri
    Sep 10, 2008
  4. KsAdmin

    Formatting the results of a calculation

    KsAdmin, Jul 6, 2004, in forum: Javascript
    Replies:
    22
    Views:
    258
    Dr John Stockton
    Jul 7, 2004
  5. Replies:
    2
    Views:
    205
Loading...

Share This Page