64-bit integers in network byte-order

Discussion in 'Ruby' started by Francis Cianfrocca, Sep 22, 2006.

  1. All,
    does Ruby have a way to convert 64-bit integers to and from network
    order in a platform-independent way?

    For 32-bit ints, I can use [value].pack("N"). The conversion operator
    for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
    this is because the underlying byte-order converters (htons/htonl) don't
    have a 64-bit version on 32-bit platforms.)

    --
    Posted via http://www.ruby-forum.com/.
    Francis Cianfrocca, Sep 22, 2006
    #1
    1. Advertising

  2. On 9/22/06, Francis Cianfrocca <> wrote:
    > All,
    > does Ruby have a way to convert 64-bit integers to and from network
    > order in a platform-independent way?
    >
    > For 32-bit ints, I can use [value].pack("N"). The conversion operator
    > for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
    > this is because the underlying byte-order converters (htons/htonl) don't
    > have a 64-bit version on 32-bit platforms.)
    >
    > --
    > Posted via http://www.ruby-forum.com/.
    >


    Hi Francis,

    A quick scan of the Ruby source tells me that you'll have to implement
    it yourself. Network byte order is big-endian so you should be able to
    do it like this:

    [val >> 32, val & 0xFFFFFFFF].pack("NN")

    At least I think that is correct.

    Cheers,
    Dave
    David Balmain, Sep 22, 2006
    #2
    1. Advertising

  3. Francis Cianfrocca

    Guest

    On Fri, 22 Sep 2006, Francis Cianfrocca wrote:

    > All,
    > does Ruby have a way to convert 64-bit integers to and from network
    > order in a platform-independent way?
    >
    > For 32-bit ints, I can use [value].pack("N"). The conversion operator
    > for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
    > this is because the underlying byte-order converters (htons/htonl) don't
    > have a 64-bit version on 32-bit platforms.)


    i don't know the answer, but maybe a hack?

    harp:~ > cat a.rb
    class String
    LITTLE_ENDIAN = ([42].pack('i')[0] == 42)
    BIG_ENDIAN = !LITTLE_ENDIAN

    def hton!() BIG_ENDIAN ? self : replace(reverse) end
    def hton() dup.hton! end
    def noth!() BIG_ENDIAN ? self : replace(reverse) end
    def noth() dup.noth! end
    end

    forty_two = [42].pack('Q')
    p forty_two

    forty_two = [42].pack('Q').hton!
    p forty_two



    harp:~ > ruby a.rb
    "*\000\000\000\000\000\000\000"
    "\000\000\000\000\000\000\000*"


    also, maybe joel's bitstruct package

    http://redshift.sourceforge.net/bit-struct/doc/index.html

    regards.

    -a
    --
    in order to be effective truth must penetrate like an arrow - and that is
    likely to hurt. -- wei wu wei
    , Sep 22, 2006
    #3
  4. wrote:
    ...
    > def hton!() BIG_ENDIAN ? self : replace(reverse) end


    replace(reverse) is no different from reverse!, is it?

    --
    vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
    Joel VanderWerf, Sep 22, 2006
    #4
  5. On 9/22/06, Francis Cianfrocca <> wrote:
    > All,
    > does Ruby have a way to convert 64-bit integers to and from network
    > order in a platform-independent way?
    >
    > For 32-bit ints, I can use [value].pack("N"). The conversion operator
    > for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
    > this is because the underlying byte-order converters (htons/htonl) don't
    > have a 64-bit version on 32-bit platforms.)
    >


    Thanks, guys. I sort of figured I'd have to hack it. Ara, I thought of
    your approach (detecting endian-ness locally) but really hoped I could
    avoid it! (Ghostbusters: "I feel so funky.")

    Combining your approach with Dave's (and passing both through several
    additional processing stages), I end up with:

    #--------------------------------------
    BigEndian = [1].pack("s") == [1].pack("n")

    def htonq value
    BigEndian ? val : ([val].pack("Q").reverse.unpack("Q").first)
    end
    #--------------------------------------

    and of course ntohq is identical to htonq.
    Francis Cianfrocca, Sep 22, 2006
    #5
  6. On 9/22/06, <> wrote:
    > LITTLE_ENDIAN = ([42].pack('i')[0] == 42)
    > BIG_ENDIAN = !LITTLE_ENDIAN



    Sorry, I really can't resist this. Do you know who the big-endians and
    the little-endians are in literary history?
    Francis Cianfrocca, Sep 22, 2006
    #6
  7. Francis Cianfrocca

    Guest

    On Sat, 23 Sep 2006, Joel VanderWerf wrote:

    > wrote:
    > ...
    >> def hton!() BIG_ENDIAN ? self : replace(reverse) end

    >
    > replace(reverse) is no different from reverse!, is it?


    it's totally different!

    err, i forgot about 'reverse!' ;-)

    cheers.

    -a
    --
    in order to be effective truth must penetrate like an arrow - and that is
    likely to hurt. -- wei wu wei
    , Sep 22, 2006
    #7
  8. Francis Cianfrocca

    Guest

    On Sat, 23 Sep 2006, Francis Cianfrocca wrote:

    > On 9/22/06, <> wrote:
    >> LITTLE_ENDIAN = ([42].pack('i')[0] == 42)
    >> BIG_ENDIAN = !LITTLE_ENDIAN

    >
    >
    > Sorry, I really can't resist this. Do you know who the big-endians and
    > the little-endians are in literary history?


    no. do tell...

    -a
    --
    in order to be effective truth must penetrate like an arrow - and that is
    likely to hurt. -- wei wu wei
    , Sep 22, 2006
    #8
  9. On 9/22/06, <> wrote:
    > > Sorry, I really can't resist this. Do you know who the big-endians and
    > > the little-endians are in literary history?

    >
    > no. do tell...
    >


    In Gulliver's Travels, a "little-endian" prefers to crack open
    soft-boiled eggs from the little end. The little-endians and the
    big-endians are unable to settle their dispute, so they go to war. The
    story has reacquired its piquancy, now that religious war is back in
    vogue.

    Sorry for the offtopic, and thanks for your help, Ara.
    Francis Cianfrocca, Sep 22, 2006
    #9
  10. Francis Cianfrocca

    Young Hyun Guest

    On Sep 22, 2006, at 8:56 AM, Francis Cianfrocca wrote:

    > On 9/22/06, Francis Cianfrocca <> wrote:
    >> All,
    >> does Ruby have a way to convert 64-bit integers to and from network
    >> order in a platform-independent way?
    >>
    >> For 32-bit ints, I can use [value].pack("N"). The conversion operator
    >> for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
    >> this is because the underlying byte-order converters (htons/htonl)
    >> don't
    >> have a 64-bit version on 32-bit platforms.)
    >>

    >
    > Thanks, guys. I sort of figured I'd have to hack it. Ara, I thought of
    > your approach (detecting endian-ness locally) but really hoped I could
    > avoid it! (Ghostbusters: "I feel so funky.")
    >
    > Combining your approach with Dave's (and passing both through several
    > additional processing stages), I end up with:
    >
    > #--------------------------------------
    > BigEndian = [1].pack("s") == [1].pack("n")
    >
    > def htonq value
    > BigEndian ? val : ([val].pack("Q").reverse.unpack("Q").first)
    > end
    > #--------------------------------------
    >
    > and of course ntohq is identical to htonq.


    Funny, I wrote almost exactly the above code just recently. But, it
    may not be necessary. Depending on your situation, you might
    consider using the "w" (BER-compressed integer) packing directive
    instead. It guarantees that arbitrarily long integers are stored in
    a standard way (big endian). The caveats are (1) it produces
    variable-length packed strings and (2) it only supports positive
    integers. However, because BER-compressed integers are self-
    describing, in the sense that you can determine when you've reached
    the end of the encoded string, you could get around the first
    limitation (if it proves to be a limitation for your usage scenario)
    by left justifying the BER-compressed integer in a fixed field. For
    my situation, I just changed the definition of my wire protocol to
    support variable-length "w"-encoded strings and didn't bother left
    justifying.

    --Young
    Young Hyun, Sep 22, 2006
    #10
  11. On 9/22/06, Young Hyun <> wrote:
    > Funny, I wrote almost exactly the above code just recently. But, it
    > may not be necessary. Depending on your situation, you might
    > consider using the "w" (BER-compressed integer) packing directive
    > instead. It guarantees that arbitrarily long integers are stored in
    > a standard way (big endian). The caveats are (1) it produces
    > variable-length packed strings and (2) it only supports positive
    > integers. However, because BER-compressed integers are self-
    > describing, in the sense that you can determine when you've reached
    > the end of the encoded string, you could get around the first
    > limitation (if it proves to be a limitation for your usage scenario)
    > by left justifying the BER-compressed integer in a fixed field. For
    > my situation, I just changed the definition of my wire protocol to
    > support variable-length "w"-encoded strings and didn't bother left
    > justifying.
    >



    Thanks for the suggestion, but I'm having to interoperate with a wire
    protocol (AMQP) that defines certain fields as eight-octet integers in
    network-order, so I don't have a choice in the matter. Clearly
    BER-encoding has the advantage of permitting nearly-arbitrary size,
    but I think what has happened here is that the protocol designers are
    heavily Java-centric, and eight-octets-in-network-order is Java's
    native encoding for a long int.
    Francis Cianfrocca, Sep 22, 2006
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Weng Tianxiang
    Replies:
    36
    Views:
    3,412
    Brannon
    Jul 15, 2006
  2. Replies:
    9
    Views:
    938
    Juha Nieminen
    Aug 22, 2007
  3. seung

    using 64-bit integers on 32-bit machine

    seung, Sep 28, 2007, in forum: C Programming
    Replies:
    5
    Views:
    1,487
    David Thompson
    Oct 15, 2007
  4. Robert Evans
    Replies:
    7
    Views:
    282
    Joel VanderWerf
    Nov 15, 2005
  5. James Harris
    Replies:
    37
    Views:
    526
    Tim Rentsch
    Aug 8, 2013
Loading...

Share This Page