64-bit integers in network byte-order

[Francis Cianfrocca]

All,
does Ruby have a way to convert 64-bit integers to and from network
order in a platform-independent way?

For 32-bit ints, I can use [value].pack("N"). The conversion operator
for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
this is because the underlying byte-order converters (htons/htonl) don't
have a 64-bit version on 32-bit platforms.)

 

[David Balmain]

All,
does Ruby have a way to convert 64-bit integers to and from network
order in a platform-independent way?

For 32-bit ints, I can use [value].pack("N"). The conversion operator
for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
this is because the underlying byte-order converters (htons/htonl) don't
have a 64-bit version on 32-bit platforms.)

Hi Francis,

A quick scan of the Ruby source tells me that you'll have to implement
it yourself. Network byte order is big-endian so you should be able to
do it like this:

[val >> 32, val & 0xFFFFFFFF].pack("NN")

At least I think that is correct.

Cheers,
Dave

 

[ara.t.howard]

All,
does Ruby have a way to convert 64-bit integers to and from network
order in a platform-independent way?

For 32-bit ints, I can use [value].pack("N"). The conversion operator
for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
this is because the underlying byte-order converters (htons/htonl) don't
have a 64-bit version on 32-bit platforms.)

i don't know the answer, but maybe a hack?

harp:~ > cat a.rb
class String
LITTLE_ENDIAN = ([42].pack('i')[0] == 42)
BIG_ENDIAN = !LITTLE_ENDIAN

def hton!() BIG_ENDIAN ? self : replace(reverse) end
def hton() dup.hton! end
def noth!() BIG_ENDIAN ? self : replace(reverse) end
def noth() dup.noth! end
end

forty_two = [42].pack('Q')
p forty_two

forty_two = [42].pack('Q').hton!
p forty_two



harp:~ > ruby a.rb
"*\000\000\000\000\000\000\000"
"\000\000\000\000\000\000\000*"


also, maybe joel's bitstruct package

http://redshift.sourceforge.net/bit-struct/doc/index.html

regards.

-a

 

[Joel VanderWerf]

def hton!() BIG_ENDIAN ? self : replace(reverse) end

replace(reverse) is no different from reverse!, is it?

 

[Francis Cianfrocca]

All,
does Ruby have a way to convert 64-bit integers to and from network
order in a platform-independent way?

For 32-bit ints, I can use [value].pack("N"). The conversion operator
for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
this is because the underlying byte-order converters (htons/htonl) don't
have a 64-bit version on 32-bit platforms.)

Thanks, guys. I sort of figured I'd have to hack it. Ara, I thought of
your approach (detecting endian-ness locally) but really hoped I could
avoid it! (Ghostbusters: "I feel so funky.")

Combining your approach with Dave's (and passing both through several
additional processing stages), I end up with:

#--------------------------------------
BigEndian = [1].pack("s") == [1].pack("n")

def htonq value
BigEndian ? val : ([val].pack("Q").reverse.unpack("Q").first)
end
#--------------------------------------

and of course ntohq is identical to htonq.

 

[Francis Cianfrocca]

LITTLE_ENDIAN = ([42].pack('i')[0] == 42)
BIG_ENDIAN = !LITTLE_ENDIAN

Sorry, I really can't resist this. Do you know who the big-endians and
the little-endians are in literary history?

 

[ara.t.howard]

replace(reverse) is no different from reverse!, is it?

it's totally different!

err, i forgot about 'reverse!' ;-)

cheers.

-a

 

[ara.t.howard]

LITTLE_ENDIAN = ([42].pack('i')[0] == 42)
BIG_ENDIAN = !LITTLE_ENDIAN

Sorry, I really can't resist this. Do you know who the big-endians and
the little-endians are in literary history?

no. do tell...

-a

 

[Francis Cianfrocca]

no. do tell...

In Gulliver's Travels, a "little-endian" prefers to crack open
soft-boiled eggs from the little end. The little-endians and the
big-endians are unable to settle their dispute, so they go to war. The
story has reacquired its piquancy, now that religious war is back in
vogue.

Sorry for the offtopic, and thanks for your help, Ara.

 

[Young Hyun]

All,
does Ruby have a way to convert 64-bit integers to and from network
order in a platform-independent way?

For 32-bit ints, I can use [value].pack("N"). The conversion operator
for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
this is because the underlying byte-order converters (htons/htonl)
don't
have a 64-bit version on 32-bit platforms.)

Thanks, guys. I sort of figured I'd have to hack it. Ara, I thought of
your approach (detecting endian-ness locally) but really hoped I could
avoid it! (Ghostbusters: "I feel so funky.")

Combining your approach with Dave's (and passing both through several
additional processing stages), I end up with:

#--------------------------------------
BigEndian = [1].pack("s") == [1].pack("n")

def htonq value
BigEndian ? val : ([val].pack("Q").reverse.unpack("Q").first)
end
#--------------------------------------

and of course ntohq is identical to htonq.

Funny, I wrote almost exactly the above code just recently. But, it
may not be necessary. Depending on your situation, you might
consider using the "w" (BER-compressed integer) packing directive
instead. It guarantees that arbitrarily long integers are stored in
a standard way (big endian). The caveats are (1) it produces
variable-length packed strings and (2) it only supports positive
integers. However, because BER-compressed integers are self-
describing, in the sense that you can determine when you've reached
the end of the encoded string, you could get around the first
limitation (if it proves to be a limitation for your usage scenario)
by left justifying the BER-compressed integer in a fixed field. For
my situation, I just changed the definition of my wire protocol to
support variable-length "w"-encoded strings and didn't bother left
justifying.

--Young

 

[Francis Cianfrocca]

Funny, I wrote almost exactly the above code just recently. But, it
may not be necessary. Depending on your situation, you might
consider using the "w" (BER-compressed integer) packing directive
instead. It guarantees that arbitrarily long integers are stored in
a standard way (big endian). The caveats are (1) it produces
variable-length packed strings and (2) it only supports positive
integers. However, because BER-compressed integers are self-
describing, in the sense that you can determine when you've reached
the end of the encoded string, you could get around the first
limitation (if it proves to be a limitation for your usage scenario)
by left justifying the BER-compressed integer in a fixed field. For
my situation, I just changed the definition of my wire protocol to
support variable-length "w"-encoded strings and didn't bother left
justifying.

Thanks for the suggestion, but I'm having to interoperate with a wire
protocol (AMQP) that defines certain fields as eight-octet integers in
network-order, so I don't have a choice in the matter. Clearly
BER-encoding has the advantage of permitting nearly-arbitrary size,
but I think what has happened here is that the protocol designers are
heavily Java-centric, and eight-octets-in-network-order is Java's
native encoding for a long int.