uint32_t 0x80000000 unpacked as 0x7FFFFFFF

Discussion in 'Perl Misc' started by A. Farber, Apr 4, 2009.

  1. A. Farber

    A. Farber Guest

    Hello,

    I'm using perl 5.8.8 and gcc 3.3.5 of OpenBSD 4.3/i386
    to unpack a C-structure received over a unix pipe
    (where event_type is enum and SID_LEN is 32):

    typedef struct request_header {
    char sid[SID_LEN + 1];
    event_type event;
    time_t modified;
    uint32_t arg;
    } request_header;

    I try to unpack it this way:

    ($self->{SID}, $self->{EVENT},
    $self->{MOD}, $self->{ARG}) =
    unpack 'A32x4L3', $self->{REQUEST};

    It mostly works well, but for the ARG 0x80000000
    I wrongly get 0x7FFFFFFF (which is same but negated?)

    Could someone please advise me?

    This signed/unsigned binary stuff is so confusing :-(
    I tried to read perlpacktut several times, never got it 100%.

    Thank you
    Alex
     
    A. Farber, Apr 4, 2009
    #1
    1. Advertising

  2. A. Farber wrote:
    >
    > I'm using perl 5.8.8 and gcc 3.3.5 of OpenBSD 4.3/i386
    > to unpack a C-structure received over a unix pipe
    > (where event_type is enum and SID_LEN is 32):
    >
    > typedef struct request_header {
    > char sid[SID_LEN + 1];
    > event_type event;
    > time_t modified;
    > uint32_t arg;
    > } request_header;


    char is a built-in C type that is defined by the standard but
    event_type, time_t and uint32_t may be a macro or typedef somewhere in
    your program and you have to determine what C type they represent in
    order to unpack them.


    > I try to unpack it this way:
    >
    > ($self->{SID}, $self->{EVENT},
    > $self->{MOD}, $self->{ARG}) =
    > unpack 'A32x4L3', $self->{REQUEST};


    char sid[SID_LEN + 1]; implies, but does not guarantee, a C "string" so
    you probably need 'Z33' to unpack it. Also, why are you skipping 4
    bytes between $self->{SID} and $self->{EVENT}?


    > It mostly works well, but for the ARG 0x80000000
    > I wrongly get 0x7FFFFFFF (which is same but negated?)


    They are two different numbers:

    $ perl -le 'print for 0x80000000, 0x7FFFFFFF'
    2147483648
    2147483647


    > Could someone please advise me?
    >
    > This signed/unsigned binary stuff is so confusing :-(


    I think that char depends on the compiler? Some define it as signed and
    some as unsigned? I have no idea whether event_type is signed or
    unsigned or what size it is. time_t, I believe, is usually a signed
    type but the size depends on how it is defined. And the 'u' at the
    front of uint32_t implies that it is an unsigned type that is 32 bits wide.


    > I tried to read perlpacktut several times, never got it 100%.




    John
    --
    Those people who think they know everything are a great
    annoyance to those of us who do. -- Isaac Asimov
     
    John W. Krahn, Apr 4, 2009
    #2
    1. Advertising

  3. A. Farber <> wrote:
    > I'm using perl 5.8.8 and gcc 3.3.5 of OpenBSD 4.3/i386
    > to unpack a C-structure received over a unix pipe
    > (where event_type is enum and SID_LEN is 32):


    > typedef struct request_header {
    > char sid[SID_LEN + 1];
    > event_type event;
    > time_t modified;
    > uint32_t arg;
    > } request_header;


    > I try to unpack it this way:


    > ($self->{SID}, $self->{EVENT},
    > $self->{MOD}, $self->{ARG}) =
    > unpack 'A32x4L3', $self->{REQUEST};


    > It mostly works well, but for the ARG 0x80000000
    > I wrongly get 0x7FFFFFFF (which is same but negated?)


    I hope you realize that this is extremely system dependend.
    The sizes of at least 'event_type' and 'time_t' depend on
    the system you're using as well as the number of padding
    bytes that may exist between the members of the structure
    (that may even be depend on the C compiler you're using!).

    And 0x80000000 and 0x7FFFFFFF are different uint32_t numbers,
    the latter being 1 less than the former.

    As John Kramer already pointed out, for the 33 byte long
    string you need "Z33" to unpack it. Due to padding bytes
    between the 'sid' and 'event' member the "x3" is needed
    (as long as there are three bytes of padding).

    Things ain't getting simpler from here. If I am not
    mistaken, an enum value must be an int. So to unpack
    it using "i" seems to be the best choice since it
    should pick the right size automatically.

    Next you've got to check if there are padding bytes following
    the 'event_type' member and, if necessary skip them with enough
    "x" - on my machine there are none.

    'time_t' is even worse since al the C standard says about that
    type is that it's "arithmetic type capable of representing times".
    So it could be a signed or unsigned int or long but as well a
    floating point type. All you can do is try to determine from
    your header files what it is on your system. On mine (which
    is x64) it seems to be a signed 64-bit integer value. Thus
    on my machine I have to use "q" for unpack() - but that may
    be different on a 32-bit system!

    Now again padding bytes may follow... None so on my machine.

    The final 'uint32_t' is simple since here only "L" is to be
    considered.

    Taking it all together I need on my machine

    unpack "Z33x3iqL", $self->{ REQUEST };

    to pick the data apart. It may need something else on yours.

    That's why it is normally not a good idea to send structures
    in binary form around since it's layout and the endian-ness
    and sizes of the members might differ from machine to machine
    (and even compiler to compiler).

    Regards, Jens
    --
    \ Jens Thoms Toerring ___
    \__________________________ http://toerring.de
     
    Jens Thoms Toerring, Apr 4, 2009
    #3
  4. A. Farber

    A. Farber Guest

    Hello,

    I've prepared a simple test case and
    found the real reason for my problem -
    the argument for atol() was out of range.

    I'll probably switch to strtol()
    in the C part of my C+Perl program.

    Yes, I was using A32x4 because the terminating
    null takes 4 bytes (because of padding).

    $ cat test-case.c
    #include <stdio.h>
    #include <unistd.h>

    #define SID "01234567890123456789012345678901"
    #define SID_LEN 32

    typedef enum event_type {
    ALIVE = 1 << 0,
    CHAT = 1 << 1,
    JOIN = 1 << 2,
    LOBBY = 1 << 3,
    INDEX = 1 << 4,
    END = 1 << 5,
    EXIT = 1 << 6
    } event_type;

    typedef struct request_header {
    char sid[SID_LEN + 1];
    event_type event;
    time_t modified;
    uint32_t arg;
    } request_header;


    int main() {
    request_header req;

    strcpy(req.sid, SID);
    req.event = INDEX;
    req.modified = 0;
    /* req.arg = 0x80000000; */
    req.arg = atol("2147483648");

    return sizeof(req) != write(STDOUT_FILENO, &req, sizeof(req));
    }

    $ cat test-case.pl
    #!/usr/bin/perl -w

    use strict;
    use Data::Dumper;

    my ($req, $href);

    $req = qx(./test-case);

    ($href->{SID}, $href->{EVENT}, $href->{MOD}, $href->{ARG}) =
    unpack 'A32x4L3', $req;

    print Dumper($href);
    printf "%x\n", $href->{ARG};

    $ ./test-case.pl
    $VAR1 = {
    'SID' => '01234567890123456789012345678901',
    'MOD' => 0,
    'EVENT' => 16,
    'ARG' => 2147483647
    };
    7fffffff

    $ perl -v
    This is perl, v5.8.8 built for i386-openbsd

    $ gcc -v
    gcc version 3.3.5 (propolice)


    Thanks for comments
    Alex
     
    A. Farber, Apr 4, 2009
    #4
  5. A. Farber

    A. Farber Guest

    On 4 Apr., 13:20, (Jens Thoms Toerring) wrote:
    > I hope you realize that this is extremely system dependend.


    This isn't too bad in my case, because it is
    an Apache-module in C talking over a Unix-pipe to
    a daemon in Perl, i.e. both run at the same machine...

    Regards
    Alex
     
    A. Farber, Apr 4, 2009
    #5
  6. A. Farber <> wrote:
    > On 4 Apr., 13:20, (Jens Thoms Toerring) wrote:
    > > I hope you realize that this is extremely system dependend.


    > This isn't too bad in my case, because it is
    > an Apache-module in C talking over a Unix-pipe to
    > a daemon in Perl, i.e. both run at the same machine...


    Ok. It's just that the same program wouldn't work on my 64-bit
    machine. And another point: for the enumerated type you should
    not unpack() for an unsigned value, they're definitely signed.
    So better use either "l" or "i" instead of "L" for that one.

    Regards, Jens
    --
    \ Jens Thoms Toerring ___
    \__________________________ http://toerring.de
     
    Jens Thoms Toerring, Apr 4, 2009
    #6
  7. A. Farber <> wrote:
    > Hello,


    > I've prepared a simple test case and
    > found the real reason for my problem -
    > the argument for atol() was out of range.


    > I'll probably switch to strtol()
    > in the C part of my C+Perl program.


    > Yes, I was using A32x4 because the terminating
    > null takes 4 bytes (because of padding).


    > $ cat test-case.c
    > #include <stdio.h>
    > #include <unistd.h>


    > #define SID "01234567890123456789012345678901"
    > #define SID_LEN 32


    > typedef enum event_type {
    > ALIVE = 1 << 0,
    > CHAT = 1 << 1,
    > JOIN = 1 << 2,
    > LOBBY = 1 << 3,
    > INDEX = 1 << 4,
    > END = 1 << 5,
    > EXIT = 1 << 6
    > } event_type;


    > typedef struct request_header {
    > char sid[SID_LEN + 1];
    > event_type event;
    > time_t modified;
    > uint32_t arg;
    > } request_header;


    > int main() {
    > request_header req;


    > strcpy(req.sid, SID);
    > req.event = INDEX;
    > req.modified = 0;
    > /* req.arg = 0x80000000; */
    > req.arg = atol("2147483648");


    Here you try to convert to a signed long (that's what the
    return type of atol() is). But that number (2^31 or, in
    hex, 0x80000000) doesn't fit into a 32-bit long (which
    seems to be what you have on your machine), and thus
    atol() returns LONG_MAX, which is 2147483647 (0x7FFFFFFFF)
    on your machine.

    Using strtol() wouldn't change that since it also will
    return LONG_MAX on too large a number. You better use
    the strtoul() function instead (uint32_t and unsigned
    long being obviously the same on your machine).

    Regards, Jens
    --
    \ Jens Thoms Toerring ___
    \__________________________ http://toerring.de
     
    Jens Thoms Toerring, Apr 4, 2009
    #7
  8. A. Farber

    A. Farber Guest

    On 4 Apr., 14:48, (Jens Thoms Toerring) wrote:
    > Using strtol() wouldn't change that since it also will
    > return LONG_MAX on too large a number. You better use
    > the strtoul() function instead (uint32_t and unsigned
    > long being obviously the same on your machine).


    gcc says:

    module/mod_pref.c:74: warning: implicit declaration of function
    `strtoul_is_not_a_portable_function_use_strtol_instead'

    Regards
    Alex
     
    A. Farber, Apr 4, 2009
    #8
  9. A. Farber <> wrote:
    > On 4 Apr., 14:48, (Jens Thoms Toerring) wrote:
    > > Using strtol() wouldn't change that since it also will
    > > return LONG_MAX on too large a number. You better use
    > > the strtoul() function instead (uint32_t and unsigned
    > > long being obviously the same on your machine).


    > gcc says:


    > module/mod_pref.c:74: warning: implicit declaration of function
    > `strtoul_is_not_a_portable_function_use_strtol_instead'


    That's rather surprising since strtoul() is required by
    the C standard to exist, and that since 20 years now.
    Did you include <stdlib.h>, where it's declared? And
    then my version of gcc (4.3.2) doesn't utter any com-
    plaints about the use of strtoul() even when invoked
    with

    gcc -std=c89 -W -Wall -ansi -pedantic

    A web search for the warning seems to indicate that
    some Apache header file may result in this nonsense
    getting spit out if strtoul() isn't declared (i.e.
    if you forgot to include <stdlib.h>). I am at a loss
    to understand what that is supposed to be good for
    (except that it's, of course, a bad idea to try to
    use the function without a declaration in scope).

    Regards, Jebs
    --
    \ Jens Thoms Toerring ___
    \__________________________ http://toerring.de
     
    Jens Thoms Toerring, Apr 4, 2009
    #9
  10. Jens Thoms Toerring wrote:
    >
    > As John Kramer already pointed out,


    John Kramer?


    John
    --
    Those people who think they know everything are a great
    annoyance to those of us who do. -- Isaac Asimov
     
    John W. Krahn, Apr 4, 2009
    #10
  11. John W. Krahn <> wrote:
    > Jens Thoms Toerring wrote:
    >>
    >> As John Kramer already pointed out,

    >
    > John Kramer?



    Maybe he meant Cosmo Kramer?


    --
    Tad McClellan
    email: perl -le "print scalar reverse qq/moc.noitatibaher\100cmdat/"
     
    Tad J McClellan, Apr 4, 2009
    #11
  12. John W. Krahn <> wrote:
    > Jens Thoms Toerring wrote:
    > >
    > > As John Kramer already pointed out,


    > John Kramer?


    Sorry, I meant you, of course. I guess I have to
    get my glasses checked...
    Regards, Jens
    --
    \ Jens Thoms Toerring ___
    \__________________________ http://toerring.de
     
    Jens Thoms Toerring, Apr 4, 2009
    #12
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    9
    Views:
    6,978
  2. Daniel Rudy
    Replies:
    15
    Views:
    1,410
    Keith Thompson
    Apr 10, 2006
  3. Giuseppe:G:
    Replies:
    1
    Views:
    481
    Giuseppe:G:
    Jul 12, 2008
  4. Francois Grieu

    [C99] how to test if type uint32_t exists?

    Francois Grieu, Feb 19, 2010, in forum: C Programming
    Replies:
    10
    Views:
    4,793
    Francois Grieu
    Feb 20, 2010
  5. Mr Magpie
    Replies:
    7
    Views:
    131
    Mr Magpie
    May 1, 2007
Loading...

Share This Page