pragma pack(1) on struct contains object

Discussion in 'C++' started by Steven Woody, Jul 12, 2008.

  1. Steven Woody

    Steven Woody Guest

    In our project, one used the following struct

    #pragma pack(1)
    struct Foo {
    uint8_t b1;
    uint8_t b2;
    uint8_t b3;
    std::vector<uint8_t> buf;
    ...
    };
    #pragma pack()

    this work on PC, but not on ARM target using a gcc/g++ cross
    compiler. The problem is, on ARM, after defined a variable of Foo
    and want to access a method of buf, such as

    Foo foo;
    foo.buf.resize(100);

    the program will exit. I guss it's a segment failure.


    Can you experts explain for me the reason? I know, it is suspious
    that why this guy use pack(1) on this kind of struct, but we can just
    ignore it and focus on the underlying reason why the usage cause the
    failure.

    Thanks in advance.

    -
    narke
     
    Steven Woody, Jul 12, 2008
    #1
    1. Advertising

  2. Steven Woody

    peter koch Guest

    On 12 Jul., 12:26, Steven Woody <> wrote:
    > In our project, one used the following struct
    >
    > #pragma pack(1)
    > struct Foo {
    >   uint8_t   b1;
    >   uint8_t   b2;
    >   uint8_t   b3;
    >   std::vector<uint8_t>  buf;
    >   ...};
    >
    > #pragma pack()
    >
    > this work on PC, but not on ARM target using a gcc/g++ cross
    > compiler.  The problem is, on ARM, after defined a variable of  Foo
    > and want to access a method of buf, such as
    >
    >     Foo foo;
    >     foo.buf.resize(100);
    >
    > the program will exit.  I guss it's a segment failure.
    >
    > Can you experts explain for me the reason?  I know, it is  suspious
    > that why this guy use pack(1) on this kind of struct, but we can just
    > ignore it and focus on the underlying reason why the usage cause the
    > failure.


    Because some processors require that certain types are properly
    aligned. So stop packing your structures: even on x86 architectures,
    packed structures (although allowed) cause the code to run much more
    slowly.

    /Peter
     
    peter koch, Jul 12, 2008
    #2
    1. Advertising

  3. Steven Woody

    Rolf Magnus Guest

    Steven Woody wrote:

    > In our project, one used the following struct
    >
    > #pragma pack(1)
    > struct Foo {
    > uint8_t b1;
    > uint8_t b2;
    > uint8_t b3;
    > std::vector<uint8_t> buf;
    > ...
    > };
    > #pragma pack()
    >
    > this work on PC, but not on ARM target using a gcc/g++ cross
    > compiler. The problem is, on ARM, after defined a variable of Foo
    > and want to access a method of buf, such as
    >
    > Foo foo;
    > foo.buf.resize(100);
    >
    > the program will exit. I guss it's a segment failure.
    >
    >
    > Can you experts explain for me the reason?


    Because you enforce mis-alingment. And while PCs can still access
    mis-algined data (though at reduced speed), an ARM CPU simply can't.

    > I know, it is suspious that why this guy use pack(1) on this kind of
    > struct, but we can just ignore it and focus on the underlying reason why
    > the usage cause the failure.


    Well that reason is at the same time the reason why this looks suspicious,
    since it's also the reason why you shouldn't do that in the first place.
     
    Rolf Magnus, Jul 12, 2008
    #3
  4. Steven Woody

    Steven Woody Guest

    On Jul 13, 1:25 am, Rolf Magnus <> wrote:
    > Steven Woody wrote:
    > > In our project, one used the following struct

    >
    > > #pragma pack(1)
    > > struct Foo {
    > > uint8_t b1;
    > > uint8_t b2;
    > > uint8_t b3;
    > > std::vector<uint8_t> buf;
    > > ...
    > > };
    > > #pragma pack()

    >
    > > this work on PC, but not on ARM target using a gcc/g++ cross
    > > compiler. The problem is, on ARM, after defined a variable of Foo
    > > and want to access a method of buf, such as

    >
    > > Foo foo;
    > > foo.buf.resize(100);

    >
    > > the program will exit. I guss it's a segment failure.

    >
    > > Can you experts explain for me the reason?

    >
    > Because you enforce mis-alingment. And while PCs can still access
    > mis-algined data (though at reduced speed), an ARM CPU simply can't.
    >
    > > I know, it is suspious that why this guy use pack(1) on this kind of
    > > struct, but we can just ignore it and focus on the underlying reason why
    > > the usage cause the failure.

    >
    > Well that reason is at the same time the reason why this looks suspicious,
    > since it's also the reason why you shouldn't do that in the first place.



    Could you explain? Thanks.

    And, I am thinking another question: whether using packing a struct is
    itself a bad notion at all. You know, we are writting a network
    program, other members in the team tends to define a structure for
    every kind of packet come in/out from/to the network, because there is
    no padding in packets in the form of byte-stream, so "#pragma pack(1)"
    or "__attribute__((packed))" was used almost everywhere. I personally
    dont' prefer this method, I just do the packet parsing and framing in
    a character by character way, hence don't mapping packets to any in-
    memory structure. Do these two kind of programming style imply
    anything? What kind do you prefer?

    Thanks in advance.
     
    Steven Woody, Jul 13, 2008
    #4
  5. Steven Woody

    Ian Collins Guest

    Steven Woody wrote:
    > On Jul 13, 1:25 am, Rolf Magnus <> wrote:
    >> Steven Woody wrote:
    >>> In our project, one used the following struct
    >>> #pragma pack(1)
    >>> struct Foo {
    >>> uint8_t b1;
    >>> uint8_t b2;
    >>> uint8_t b3;
    >>> std::vector<uint8_t> buf;
    >>> ...
    >>> };
    >>> #pragma pack()
    >>> this work on PC, but not on ARM target using a gcc/g++ cross
    >>> compiler. The problem is, on ARM, after defined a variable of Foo
    >>> and want to access a method of buf, such as
    >>> Foo foo;
    >>> foo.buf.resize(100);
    >>> the program will exit. I guss it's a segment failure.
    >>> Can you experts explain for me the reason?

    >> Because you enforce mis-alingment. And while PCs can still access
    >> mis-algined data (though at reduced speed), an ARM CPU simply can't.
    >>
    >>> I know, it is suspious that why this guy use pack(1) on this kind of
    >>> struct, but we can just ignore it and focus on the underlying reason why
    >>> the usage cause the failure.

    >> Well that reason is at the same time the reason why this looks suspicious,
    >> since it's also the reason why you shouldn't do that in the first place.

    >
    >
    > Could you explain? Thanks.
    >

    The quote the paragraph before:

    >> Because you enforce mis-alingment. And while PCs can still access
    >> mis-algined data (though at reduced speed), an ARM CPU simply can't.


    > And, I am thinking another question: whether using packing a struct is
    > itself a bad notion at all.


    It is if you want portable code. Some common processors simply don't
    support it.

    > You know, we are writting a network
    > program, other members in the team tends to define a structure for
    > every kind of packet come in/out from/to the network, because there is
    > no padding in packets in the form of byte-stream, so "#pragma pack(1)"
    > or "__attribute__((packed))" was used almost everywhere.


    So they don't care about portability? Not only is packing an issue,
    there's byte order as well.

    > I personally
    > dont' prefer this method, I just do the packet parsing and framing in
    > a character by character way, hence don't mapping packets to any in-
    > memory structure. Do these two kind of programming style imply
    > anything? What kind do you prefer?
    >

    The latter.

    --
    Ian Collins.
     
    Ian Collins, Jul 13, 2008
    #5
  6. Steven Woody

    Ian Collins Guest

    Ian Collins wrote:
    > Steven Woody wrote:
    >
    >> You know, we are writting a network
    >> program, other members in the team tends to define a structure for
    >> every kind of packet come in/out from/to the network, because there is
    >> no padding in packets in the form of byte-stream, so "#pragma pack(1)"
    >> or "__attribute__((packed))" was used almost everywhere.

    >
    > So they don't care about portability? Not only is packing an issue,
    > there's byte order as well.
    >
    >> I personally
    >> dont' prefer this method, I just do the packet parsing and framing in
    >> a character by character way, hence don't mapping packets to any in-
    >> memory structure. Do these two kind of programming style imply
    >> anything? What kind do you prefer?
    >>

    > The latter.
    >

    I should clarify, I do use structures for packet types, but I don't just
    pull them off the wire. I use a binary stream class to serialise the
    structures and manage reading from and writing to the network.

    --
    Ian Collins.
     
    Ian Collins, Jul 13, 2008
    #6
  7. Steven Woody

    Steven Woody Guest

    On Jul 13, 12:12 pm, Ian Collins <> wrote:
    > Ian Collins wrote:
    > > Steven Woody wrote:

    >
    > >> You know, we are writting a network
    > >> program, other members in the team tends to define a structure for
    > >> every kind of packet come in/out from/to the network, because there is
    > >> no padding in packets in the form of byte-stream, so "#pragma pack(1)"
    > >> or "__attribute__((packed))" was used almost everywhere.

    >
    > > So they don't care about portability? Not only is packing an issue,
    > > there's byte order as well.

    >
    > >> I personally
    > >> dont' prefer this method, I just do the packet parsing and framing in
    > >> a character by character way, hence don't mapping packets to any in-
    > >> memory structure. Do these two kind of programming style imply
    > >> anything? What kind do you prefer?

    >
    > > The latter.

    >
    > I should clarify, I do use structures for packet types, but I don't just
    > pull them off the wire. I use a binary stream class to serialise the
    > structures and manage reading from and writing to the network.
    >
    > --
    > Ian Collins.


    This is indeed what I did though I dont use an explicit stream class
    as yours, but I transfer merely byte stream (std::vector<uint8_t> for
    example) between upper and lower software layers, and I write utility
    function to encode any internal structure to a std::vector<uint8_t>.
    For reading packet from byte stream, serialization is not even enough,
    since valid packets may come between noise, that means I have to parse
    and recognize them from the stream, hence, an Finite-State-Machine is
    always employed, do this way means I read in characters one by one and
    pop up a valid packet to upper layer if any found.

    In your previous post, I thing I don't fully understand the term 'mis-
    alignment': "Because you enforce mis-alingment. And while PCs can
    still access
    >> mis-algined data (though at reduced speed), an ARM CPU simply can't. ", by the term 'mis-alignment', did you mean any manually treatment (such as puting a #pragma pack(1) or __atttritute__((packed)) ) putting on a structure? If so, can I understand this way: mis-alignment means any method which cause the default layout or size of structure changed? If I am right, it seems I can not explain the fact that mis-alignment did not rise any problem on my ARM if the structure does not contains any non-POD type.


    PS. Answer a question you asked: there is not byte-ordering problem
    in the project, you know, PC and ARM both use little-endian which is
    same as byte order used in our communication protocol.
     
    Steven Woody, Jul 13, 2008
    #7
  8. Steven Woody

    Ian Collins Guest

    Steven Woody wrote:
    >

    *Please* don't quote sigs
    >
    >
    > In your previous post, I thing I don't fully understand the term 'mis-
    > alignment':


    Most, if not all processors prefer types to be aligned to their natural
    boundary, say 4 bytes for an int. When an int value is not aligned on a
    4 byte boundary, the processor may require one or two extra bus cycles
    to read the value form memory.

    Some processors (Sparc for example) *require* types to be aligned to
    their natural boundary. They will generate a hardware trap it you
    attempt a misaligned access.

    --
    Ian Collins.
     
    Ian Collins, Jul 13, 2008
    #8
  9. Steven Woody

    James Kanze Guest

    On Jul 13, 5:00 am, Steven Woody <> wrote:
    > On Jul 13, 1:25 am, Rolf Magnus <> wrote:
    > > Steven Woody wrote:
    > > > I know, it is suspious that why this guy use pack(1) on
    > > > this kind of struct, but we can just ignore it and focus
    > > > on the underlying reason why the usage cause the failure.


    > > Well that reason is at the same time the reason why this
    > > looks suspicious, since it's also the reason why you
    > > shouldn't do that in the first place.


    > Could you explain? Thanks.


    Because that's what pack does, as Rolf explained. Practically
    speaking, just about any use of pack is suspicious. About the
    only case I can think of where you might use it would be in
    driver software, in the kernel, and even there, I've never found
    it useful.

    > And, I am thinking another question: whether using packing a
    > struct is itself a bad notion at all.


    Very.

    > You know, we are writting a network program, other members in
    > the team tends to define a structure for every kind of packet
    > come in/out from/to the network, because there is no padding
    > in packets in the form of byte-stream, so "#pragma pack(1)" or
    > "__attribute__((packed))" was used almost everywhere.


    Which is just stupid. Packing is not the only aspect which has
    to be considered. It makes a lot of sense to defined structures
    (or even better, classes) for network packets, but you still
    have to marshal and unmarshal, at the byte level, in order to
    respect variations in the representation. (The most frequent
    variation for integral types is byte order and size, although
    there are machines which don't use 2's complement. For floating
    point, of course, there's even more variation.)

    > I personally dont' prefer this method, I just do the packet
    > parsing and framing in a character by character way, hence
    > don't mapping packets to any in- memory structure.


    Which is the only way which really works.

    > Do these two kind of programming style imply
    > anything? What kind do you prefer?


    Yes. The use of a byte dump and pack means that the authors
    were incompetent.

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
     
    James Kanze, Jul 13, 2008
    #9
  10. Steven Woody

    Steven Woody Guest

    On Jul 13, 4:55 pm, James Kanze <> wrote:
    > On Jul 13, 5:00 am, Steven Woody <> wrote:
    >
    > > On Jul 13, 1:25 am, Rolf Magnus <> wrote:
    > > > Steven Woody wrote:
    > > > > I know, it is suspious that why this guy use pack(1) on
    > > > > this kind of struct, but we can just ignore it and focus
    > > > > on the underlying reason why the usage cause the failure.
    > > > Well that reason is at the same time the reason why this
    > > > looks suspicious, since it's also the reason why you
    > > > shouldn't do that in the first place.

    > > Could you explain? Thanks.

    >
    > Because that's what pack does, as Rolf explained. Practically
    > speaking, just about any use of pack is suspicious. About the
    > only case I can think of where you might use it would be in
    > driver software, in the kernel, and even there, I've never found
    > it useful.
    >
    > > And, I am thinking another question: whether using packing a
    > > struct is itself a bad notion at all.

    >
    > Very.
    >
    > > You know, we are writting a network program, other members in
    > > the team tends to define a structure for every kind of packet
    > > come in/out from/to the network, because there is no padding
    > > in packets in the form of byte-stream, so "#pragma pack(1)" or
    > > "__attribute__((packed))" was used almost everywhere.

    >
    > Which is just stupid. Packing is not the only aspect which has
    > to be considered. It makes a lot of sense to defined structures
    > (or even better, classes) for network packets, but you still
    > have to marshal and unmarshal, at the byte level, in order to
    > respect variations in the representation. (The most frequent
    > variation for integral types is byte order and size, although
    > there are machines which don't use 2's complement. For floating
    > point, of course, there's even more variation.)
    >
    > > I personally dont' prefer this method, I just do the packet
    > > parsing and framing in a character by character way, hence
    > > don't mapping packets to any in- memory structure.

    >
    > Which is the only way which really works.
    >
    > > Do these two kind of programming style imply
    > > anything? What kind do you prefer?

    >
    > Yes. The use of a byte dump and pack means that the authors
    > were incompetent.
    >
    > --
    > James Kanze (GABI Software) email:
    > Conseils en informatique orientée objet/
    > Beratung in objektorientierter Datenverarbeitung
    > 9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34


    Thank you, Rolf, Ian and James. Your kindly shared opinions made me
    understand the topic more clear than ever.
     
    Steven Woody, Jul 13, 2008
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. JustSomeGuy

    how does pragma pack work?

    JustSomeGuy, Dec 9, 2003, in forum: C++
    Replies:
    1
    Views:
    3,725
    Pete Becker
    Dec 9, 2003
  2. Tim Jones
    Replies:
    0
    Views:
    405
    Tim Jones
    Jan 31, 2004
  3. Jimmy

    Why #pragma pack not take effect?

    Jimmy, Jul 3, 2007, in forum: C Programming
    Replies:
    5
    Views:
    796
    Kenny McCormack
    Jul 4, 2007
  4. Boltar

    Question about using #pragma pack

    Boltar, Mar 20, 2008, in forum: C Programming
    Replies:
    7
    Views:
    417
    Keith Thompson
    Mar 20, 2008
  5. Fedor Rodikov
    Replies:
    6
    Views:
    694
    James Kanze
    Sep 2, 2011
Loading...

Share This Page