Re: Strange array access in two executables compiled with different compilers

Discussion in 'C++' started by Juha Nieminen, Sep 21, 2012.

  1. Fab <> wrote:
    > This makes absolute no sense to me, can somebody think of something why
    > this could happen??


    It makes perfect sense. The compiler doesn't add any boundary checks to
    the array accessing. Anything can happen.
     
    Juha Nieminen, Sep 21, 2012
    #1
    1. Advertising

  2. Juha Nieminen

    Stuart Guest

    Re: Strange array access in two executables compiled with differentcompilers

    Fab wrote:
    [OP tried to access an array out of bounds, got different results on
    with different platforms]
    >> This makes absolute no sense to me, can somebody think of something why
    >> this could happen??


    On 9/21/12 Juha Nieminen wrote:
    > It makes perfect sense. The compiler doesn't add any boundary checks to
    > the array accessing. Anything can happen.


    AFAIK, Microsoft's Visual C++ compiler will surround dynamically
    allocated arrays with some safeguard padding, and when this padding is
    accessed, VC will give you some kind of runtime error. But this
    mechanism should only work for one-dimensional arrays: If you iterate
    over the first index of a multi-dimensional array, chances are high that
    one step beyond the limits of the highest dimension will refer to memory
    that lies not only beyond the memory of the array but also beyond the
    safety padding.

    However, if the array in question is non a plain old array but an array
    class, the operator[] might do the checks (I'm pretty sure that
    operator[] for ordinary arrays does not do any bounds checking except
    through the padding mechanism).

    BTW, I would be pretty surprised if GCC were not able to provide the
    same behaviour as Visual C. Possibly some kind of compiler flag?

    Regards,
    Stuart
     
    Stuart, Sep 21, 2012
    #2
    1. Advertising

  3. Stuart <> wrote:
    > AFAIK, Microsoft's Visual C++ compiler will surround dynamically
    > allocated arrays with some safeguard padding, and when this padding is
    > accessed, VC will give you some kind of runtime error. But this
    > mechanism should only work for one-dimensional arrays: If you iterate
    > over the first index of a multi-dimensional array, chances are high that
    > one step beyond the limits of the highest dimension will refer to memory
    > that lies not only beyond the memory of the array but also beyond the
    > safety padding.


    It's technically impossible for the program to detect an out-of-bounds
    access in that manner unless it specifically adds a conditional to each
    such access that checks that the index is within limits. But in that case
    you wouldn't need the padding at all (because the conditional itself
    suffices).

    (Note that the CPU and on top of that the OS supports the concept of
    a *page fault*. This means that the memory available to the program is
    mapped into virtual pages. (These virtual pages may map pretty randomly
    into physical memory and may in fact change even during the execution of
    the program and be, for example, swapped into disk. When you think you
    are traversing an array linearly, you may in fact be traversing the
    physical RAM in a pretty randomized order in terms of the physical
    addresses.) This is what causes a segmentation fault: You have tried to
    access a virtual page that's not allocated for your process. However,
    virtual pages have fixed sizes and are relatively large (usually in the
    order of kilobytes) and thus cannot be used to accurately detect
    out-of-bounds accesses of arbitrary-sized arrays. Very often if you have
    a off-by-one or such small out-of-bounds access, you will be simply
    accessing whatever else memory your program is using.)

    There are techniques employed by tools like valgrind and similar that
    pad allocations with a certain amount of extra space, put a certain bit
    pattern there and regularly check if the bit pattern has changed. This
    is usually how they detect if the process has accessed outside any
    allocated block (although I'm sure there are even more clever techniques
    for this that I'm not fully aware of.) However, compilers do not usually
    employ this kind of techniques because they slow down the program
    considerably, except perhaps in some kind of debug mode, in some compilers.

    If Visual C can indeed detect out-of-bounds accesses of raw arrays, it
    uses either conditionals (which don't help if you are accessing the array
    through a pointer) or some other technique which it only uses in debug
    mode.

    > However, if the array in question is non a plain old array but an array
    > class, the operator[] might do the checks


    Normally you would want that only when compiling in debug mode.

    > BTW, I would be pretty surprised if GCC were not able to provide the
    > same behaviour as Visual C. Possibly some kind of compiler flag?


    You can define the preprocessor macro _GLIBCXX_DEBUG to turn on boundary
    and many other sanity checks for the standard library, but it won't work
    with raw arrays nor raw pointers.
     
    Juha Nieminen, Sep 21, 2012
    #3
  4. Juha Nieminen

    Stuart Guest

    Re: Strange array access in two executables compiled with differentcompilers

    Stuart wrote:
    >> AFAIK, Microsoft's Visual C++ compiler will surround dynamically
    >> allocated arrays with some safeguard padding, and when this padding is
    >> accessed, VC will give you some kind of runtime error. But this
    >> mechanism should only work for one-dimensional arrays: If you iterate
    >> over the first index of a multi-dimensional array, chances are high that
    >> one step beyond the limits of the highest dimension will refer to memory
    >> that lies not only beyond the memory of the array but also beyond the
    >> safety padding.


    On 9/21/12 Juha Nieminen wrote:
    > It's technically impossible for the program to detect an out-of-bounds
    > access in that manner unless it specifically adds a conditional to each
    > such access that checks that the index is within limits. But in that case
    > you wouldn't need the padding at all (because the conditional itself
    > suffices).


    Checking the index is only possible if the array is actually accessed
    via the index. The most common use case is probably that the code
    iterates over the array using a pointer. In such cases, only the padding
    would catch out-of-bounds access.

    On 9/21/12 Juha Nieminen wrote:
    > There are techniques employed by tools like valgrind and similar that
    > pad allocations with a certain amount of extra space, put a certain bit
    > pattern there and regularly check if the bit pattern has changed. This
    > is usually how they detect if the process has accessed outside any
    > allocated block (although I'm sure there are even more clever techniques
    > for this that I'm not fully aware of.)

    [snip]

    I thought that the compiler did the padding, but I may be wrong. Maybe
    this was only used for DEBUG_NEW from MFC, but I can't check this out
    because I don't use VC anymore.

    Regards,
    Stuart
     
    Stuart, Sep 21, 2012
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Matthew K Jensen
    Replies:
    0
    Views:
    286
    Matthew K Jensen
    Aug 29, 2004
  2. Matthew K Jensen
    Replies:
    6
    Views:
    388
    Michael Geary
    Sep 5, 2004
  3. John
    Replies:
    0
    Views:
    278
  4. geletine

    commercial c compilers vs free c compilers

    geletine, Jul 2, 2006, in forum: C Programming
    Replies:
    33
    Views:
    1,364
  5. Clifford Stern

    Compilers no longer release executables

    Clifford Stern, Feb 19, 2009, in forum: C Programming
    Replies:
    2
    Views:
    312
    CBFalconer
    Feb 20, 2009
Loading...

Share This Page