memmove: works on C's abstraction layer? or no?

Discussion in 'C Programming' started by Kobu, Jul 22, 2006.

  1. Kobu

    Kobu Guest

    Hi,

    The addresses seens in C's "layer" aren't necessarily the same as the
    actual physical addresses in computer core (in fact, this is almost
    always true for big OSes because of memory management).

    So does this also mean that a certain number of contiguous bytes of
    memory in C's layer (say, an array of chars or ints) is not necessarily
    adjacent/contiguous bytes in core (but it probably is in big OSes)?

    If the above is correct (not necessarily the same), then is memmove
    guaranteed to make up for the differences? Say I have a char[40] array
    in a program in any hosted C compiler in the world, and I memove the 40
    bytes in the array to some other buffer in my program. Is it
    guaranteed to be contiguous (at C's level) and in the same order?

    Thanks..
    Kobu, Jul 22, 2006
    #1
    1. Advertising

  2. Kobu

    Tom St Denis Guest

    Kobu wrote:
    > Hi,
    >
    > The addresses seens in C's "layer" aren't necessarily the same as the
    > actual physical addresses in computer core (in fact, this is almost
    > always true for big OSes because of memory management).
    >
    > So does this also mean that a certain number of contiguous bytes of
    > memory in C's layer (say, an array of chars or ints) is not necessarily
    > adjacent/contiguous bytes in core (but it probably is in big OSes)?
    >
    > If the above is correct (not necessarily the same), then is memmove
    > guaranteed to make up for the differences? Say I have a char[40] array
    > in a program in any hosted C compiler in the world, and I memove the 40
    > bytes in the array to some other buffer in my program. Is it
    > guaranteed to be contiguous (at C's level) and in the same order?


    memmove() is a userspace C library function, it only "knows about" your
    processes virtual memory. So no, it doesn't matter if your physical
    memory is contiguous. That's the point of virtual memory [well among
    other goals].

    Tom
    Tom St Denis, Jul 22, 2006
    #2
    1. Advertising

  3. Kobu said:

    >
    > Hi,
    >
    > The addresses seens in C's "layer" aren't necessarily the same as the
    > actual physical addresses in computer core (in fact, this is almost
    > always true for big OSes because of memory management).
    >
    > So does this also mean that a certain number of contiguous bytes of
    > memory in C's layer (say, an array of chars or ints) is not necessarily
    > adjacent/contiguous bytes in core (but it probably is in big OSes)?


    Arrays use contiguous storage, as far as C is concerned. If the
    implementation wants to play games with that, I suppose it can, provided
    the "as-if" rule is followed.

    > If the above is correct (not necessarily the same), then is memmove
    > guaranteed to make up for the differences?


    Yes.

    > Say I have a char[40] array
    > in a program in any hosted C compiler in the world, and I memove the 40
    > bytes in the array to some other buffer in my program. Is it
    > guaranteed to be contiguous (at C's level) and in the same order?


    Yes, provided that both the source and destination are legitimate.

    --
    Richard Heathfield
    "Usenet is a strange place" - dmr 29/7/1999
    http://www.cpax.org.uk
    email: rjh at above domain (but drop the www, obviously)
    Richard Heathfield, Jul 22, 2006
    #3
  4. >The addresses seens in C's "layer" aren't necessarily the same as the
    >actual physical addresses in computer core (in fact, this is almost
    >always true for big OSes because of memory management).


    If the machine has virtual addresses and C is operating at the
    user-program level where programs use virtual addresses, then C
    operates on virtual addresses. A good virtual-memory system does
    not require you to know, or even PERMIT you to know, physical
    addresses (and they can change at any time).

    >So does this also mean that a certain number of contiguous bytes of
    >memory in C's layer (say, an array of chars or ints) is not necessarily
    >adjacent/contiguous bytes in core (but it probably is in big OSes)?


    There is no guarantee that all of an array even exists in physical
    memory simultaneously, so the issue of its being contiguous doesn't
    even apply. Also, it can change at any time. There might be only
    one page of physical memory maximum at any one time for this program.
    (performance will suck horribly, but that's a quality-of-implementation
    issue).

    >If the above is correct (not necessarily the same), then is memmove
    >guaranteed to make up for the differences? Say I have a char[40] array
    >in a program in any hosted C compiler in the world, and I memove the 40
    >bytes in the array to some other buffer in my program. Is it
    >guaranteed to be contiguous (at C's level) and in the same order?


    C (including the C library) operates on virtual addresses. A
    well-designed virtual memory system does not permit you to find out
    if the array is physically contiguous (although if it fits in a
    single page and you know the virtual memory page size, you can make
    a good guess.

    Gordon L. Burditt
    Gordon Burditt, Jul 22, 2006
    #4
  5. Kobu

    Tom St Denis Guest

    Gordon Burditt wrote:
    > >The addresses seens in C's "layer" aren't necessarily the same as the
    > >actual physical addresses in computer core (in fact, this is almost
    > >always true for big OSes because of memory management).

    >
    > If the machine has virtual addresses and C is operating at the
    > user-program level where programs use virtual addresses, then C
    > operates on virtual addresses. A good virtual-memory system does
    > not require you to know, or even PERMIT you to know, physical
    > addresses (and they can change at any time).


    <snip>

    I agree with the bulk of what you're saying. Though I disagree with
    the notion that knowing physical addresses is taboo. You can't use
    them even if you did know them. That's the point of paging and
    segmentation.

    Tom
    Tom St Denis, Jul 22, 2006
    #5
  6. >> >The addresses seens in C's "layer" aren't necessarily the same as the
    >> >actual physical addresses in computer core (in fact, this is almost
    >> >always true for big OSes because of memory management).

    >>
    >> If the machine has virtual addresses and C is operating at the
    >> user-program level where programs use virtual addresses, then C
    >> operates on virtual addresses. A good virtual-memory system does
    >> not require you to know, or even PERMIT you to know, physical
    >> addresses (and they can change at any time).

    >
    ><snip>
    >
    >I agree with the bulk of what you're saying. Though I disagree with
    >the notion that knowing physical addresses is taboo.


    >You can't use
    >them even if you did know them.


    If you know them, you can (mis)use them, and this generates bug
    reports and tech support calls for a problem that likely doesn't
    happen 99% of the time and is very difficult to consistently
    reproduce. That's what makes it taboo to get physical addresses.

    >That's the point of paging and
    >segmentation.


    Gordon L. Burditt
    Gordon Burditt, Jul 22, 2006
    #6
  7. Kobu

    Tom St Denis Guest

    Gordon Burditt wrote:
    > If you know them, you can (mis)use them, and this generates bug


    How do you use physical addresses?

    You can move data there, you can't issue DMA or other commands, etc.

    Hint: if you wanted to attack or snoop would you need to know the
    address of your victim process? You could easily snoop the entire
    physical space. So even if you didn't tell me the physical address of
    your process memory pages it wouldn't matter.

    So either you're insecure anyways, or it doesn't matter.

    Tom
    Tom St Denis, Jul 22, 2006
    #7
  8. Kobu

    Eric Sosman Guest

    Kobu wrote On 07/22/06 14:57,:
    > Hi,
    >
    > The addresses seens in C's "layer" aren't necessarily the same as the
    > actual physical addresses in computer core (in fact, this is almost
    > always true for big OSes because of memory management).


    True.

    > So does this also mean that a certain number of contiguous bytes of
    > memory in C's layer (say, an array of chars or ints) is not necessarily
    > adjacent/contiguous bytes in core (but it probably is in big OSes)?


    Yes, I think. (My only doubt is about what, exactly,
    you mean by "contiguous bytes in core." Whatever your
    definition of contiguity, it is unlikely to be preserved
    when one takes into account a memory management unit, the
    presence of various levels of physical and virtual cache,
    and the interesting fact that few if any machines nowadays
    contain any core memory at all.)

    > If the above is correct (not necessarily the same), then is memmove
    > guaranteed to make up for the differences? Say I have a char[40] array
    > in a program in any hosted C compiler in the world, and I memove the 40
    > bytes in the array to some other buffer in my program. Is it
    > guaranteed to be contiguous (at C's level) and in the same order?


    C operates at the level of the abstract machine as
    described in the Standard, and its notions of adjacency,
    of contiguity, and even of "address" pertain to that level.
    If a machine's implementation embodies additional, deeper
    levels, the C language is oblivious to them. It is the job
    of the implementation to "make it work," somehow.

    Thought question: If the first four characters of your
    array reside in DIMM0, the next sixteen in DIMM1, and the
    remaining twenty on the swap disk, is the array contiguous?

    --
    Eric Sosman, Jul 22, 2006
    #8
  9. >> If you know them, you can (mis)use them, and this generates bug
    >
    >How do you use physical addresses?


    You can use physical addresses as a poor substitute for the output
    of rand(), except the output of rand() is repeatable with a given
    seed, and physical addresses aren't.

    You can use physical addresses (assuming the existence of a
    virtual-to-physical-address translator function) to check if a large
    array is physically contiguous, and call abort() if so. The few
    occasions this triggers will be a problematic source of bug reports.

    >You can move data there,


    I know of no function that can move data to a physical address (or
    read data from a physical address) without requiring privileges on
    a virtual-memory system. Even if it's YOUR process! And I don't
    know of any standardized method of doing this regardless of privileges.

    >you can't issue DMA or other commands, etc.
    >
    >Hint: if you wanted to attack or snoop would you need to know the
    >address of your victim process? You could easily snoop the entire
    >physical space. So even if you didn't tell me the physical address of
    >your process memory pages it wouldn't matter.


    You're assuming the existence of a read-by-physical-address function.
    I don't know of any such function that does not require privileges
    to use on any OS with virtual memory.

    >So either you're insecure anyways, or it doesn't matter.


    If you permit an unprivileged read-data-by-physical-address, you're
    insecure. If you permit an unprivileged
    virtual-address-to-physical-address translator function (with *NO*
    ability to access by physical address), there is nothing useful you
    can use the info for (it is of extremely limited use for snooping),
    but anything the info is used for is going to cause trouble.

    Gordon L. Burditt
    Gordon Burditt, Jul 22, 2006
    #9
  10. "Tom St Denis" <> writes:
    > Gordon Burditt wrote:
    >> If you know them, you can (mis)use them, and this generates bug

    >
    > How do you use physical addresses?


    If a system provided a way to obtain a physical address, presumably it
    would provide a way to use it. I have no idea what that might look
    like.

    Or a buggy program could attempt to use a physical address as if it
    were a virtual address.

    Gordon: Stop snipping attributions.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
    Keith Thompson, Jul 22, 2006
    #10
  11. Kobu

    Malcolm Guest

    "Kobu" <> wrote
    >
    > The addresses seens in C's "layer" aren't necessarily the same as the
    > actual physical addresses in computer core (in fact, this is almost
    > always true for big OSes because of memory management).
    >
    > So does this also mean that a certain number of contiguous bytes of
    > memory in C's layer (say, an array of chars or ints) is not necessarily
    > adjacent/contiguous bytes in core (but it probably is in big OSes)?
    >
    > If the above is correct (not necessarily the same), then is memmove
    > guaranteed to make up for the differences? Say I have a char[40] array
    > in a program in any hosted C compiler in the world, and I memove the 40
    > bytes in the array to some other buffer in my program. Is it
    > guaranteed to be contiguous (at C's level) and in the same order?
    >

    Yes.
    Within an object, such as a structure or an array, you can compare
    addresses and they are always ordered.
    char buff[40];

    &buff[0] is always less than &buff[1] which is always less than &buff[2].
    &buff[0] + 1 will always give &buff[1].

    However the physical layout of the data on the memory chip may bear no
    relation to this order. C shields you from that.

    >

    --
    Buy my book 12 Common Atheist Arguments (refuted)
    $1.25 download or $7.20 paper, available www.lulu.com/bgy1mm
    Malcolm, Jul 22, 2006
    #11
  12. Kobu

    Coos Haak Guest

    Op Sat, 22 Jul 2006 21:36:31 +0100 schreef Malcolm:

    > &buff[0] is always less than &buff[1] which is always less than &buff[2].
    > &buff[0] + 1 will always give &buff[1].
    >
    > However the physical layout of the data on the memory chip may bear no
    > relation to this order. C shields you from that.
    >

    How could it, it's the operating system and the memory management units
    that shields the user from the physical address.
    --
    Coos
    Coos Haak, Jul 22, 2006
    #12
  13. Kobu

    Jack Klein Guest

    Jack Klein, Jul 23, 2006
    #13
  14. Kobu

    Jack Klein Guest

    Jack Klein, Jul 23, 2006
    #14
  15. Kobu

    Jack Klein Guest

    Jack Klein, Jul 23, 2006
    #15
  16. Kobu

    Joe Wright Guest

    Eric Sosman wrote:
    >
    > Kobu wrote On 07/22/06 14:57,:
    >> Hi,
    >>
    >> The addresses seens in C's "layer" aren't necessarily the same as the
    >> actual physical addresses in computer core (in fact, this is almost
    >> always true for big OSes because of memory management).

    >
    > True.
    >
    >> So does this also mean that a certain number of contiguous bytes of
    >> memory in C's layer (say, an array of chars or ints) is not necessarily
    >> adjacent/contiguous bytes in core (but it probably is in big OSes)?

    >
    > Yes, I think. (My only doubt is about what, exactly,
    > you mean by "contiguous bytes in core." Whatever your
    > definition of contiguity, it is unlikely to be preserved
    > when one takes into account a memory management unit, the
    > presence of various levels of physical and virtual cache,
    > and the interesting fact that few if any machines nowadays
    > contain any core memory at all.)
    >
    >> If the above is correct (not necessarily the same), then is memmove
    >> guaranteed to make up for the differences? Say I have a char[40] array
    >> in a program in any hosted C compiler in the world, and I memove the 40
    >> bytes in the array to some other buffer in my program. Is it
    >> guaranteed to be contiguous (at C's level) and in the same order?

    >
    > C operates at the level of the abstract machine as
    > described in the Standard, and its notions of adjacency,
    > of contiguity, and even of "address" pertain to that level.
    > If a machine's implementation embodies additional, deeper
    > levels, the C language is oblivious to them. It is the job
    > of the implementation to "make it work," somehow.
    >
    > Thought question: If the first four characters of your
    > array reside in DIMM0, the next sixteen in DIMM1, and the
    > remaining twenty on the swap disk, is the array contiguous?
    >

    Of course, because C says the elements of an array are contiguous. Our
    program doesn't know or care where the bits are.

    If there is no more core memory, how can we get a core dump? :)

    --
    Joe Wright
    "Everything should be made as simple as possible, but not simpler."
    --- Albert Einstein ---
    Joe Wright, Jul 23, 2006
    #16
  17. In article <>,
    Gordon Burditt <> wrote:

    >You can use physical addresses (assuming the existence of a
    >virtual-to-physical-address translator function) to check if a large
    >array is physically contiguous, and call abort() if so. The few
    >occasions this triggers will be a problematic source of bug reports.


    Checking for physical contiguity is by no means unknown, nor is it
    without its uses.

    In particular, it is not unknown for programs to determine whether a
    chunk of memory lies entirely within one of the systems' virtual pages,
    so as to avoid virtual page faults (which might require disk activity,
    and thus might not satisify real-time guarantees); or to optimize with
    respect to the system caching properties (e.g., to avoid a call to the
    next lower level cache); or to avoid leaking sensitive information by
    virtual of triggering or not triggering a page fault.

    For example, in strong cryptography, one needs to ensure consistant
    timing no matter whether the key matches or not, and one needs to
    ensure that one cannot gain information about -any- aspect of the
    decryption by side-channels such as whether a fault is triggered or not.

    The checking of physical contiguity might be as simple as
    checking (or knowing the fixed constant) virtual page size, together
    with the information that a single virtual page is always physically
    contiguous on the target system; combine this with some trivial
    address arithmetic to determine contiguity. One does not need to
    know the physical addresses in order to make this kind of determination.
    --
    Prototypes are supertypes of their clones. -- maplesoft
    Walter Roberson, Jul 23, 2006
    #17
  18. Kobu

    Malcolm Guest

    "Coos Haak" <> wrote in message
    news:wv8s9a6u2ww5$.fnrq4dsmoeen$...
    > Op Sat, 22 Jul 2006 21:36:31 +0100 schreef Malcolm:
    >
    >> &buff[0] is always less than &buff[1] which is always less than &buff[2].
    >> &buff[0] + 1 will always give &buff[1].
    >>
    >> However the physical layout of the data on the memory chip may bear no
    >> relation to this order. C shields you from that.
    >>

    > How could it, it's the operating system and the memory management units
    > that shields the user from the physical address.
    >

    Not always. Many C programs run on systems without operating systems.
    For instance, a lot of systems have ways of forcing data into a cache. C
    doesn't allow you to do this, you must simply trust the compiler to do a
    reasonable job for you. Generally this is a good thing, compliers are
    better than people at most types of cache management.
    --
    Buy my book 12 Common Atheist Arguments (refuted)
    $1.25 download or $7.20 paper, available www.lulu.com/bgy1mm
    Malcolm, Jul 23, 2006
    #18
  19. "Malcolm" <> writes:
    > "Coos Haak" <> wrote in message
    > news:wv8s9a6u2ww5$.fnrq4dsmoeen$...
    >> Op Sat, 22 Jul 2006 21:36:31 +0100 schreef Malcolm:
    >>
    >>> &buff[0] is always less than &buff[1] which is always less than &buff[2].
    >>> &buff[0] + 1 will always give &buff[1].
    >>>
    >>> However the physical layout of the data on the memory chip may bear no
    >>> relation to this order. C shields you from that.
    >>>

    >> How could it, it's the operating system and the memory management units
    >> that shields the user from the physical address.
    >>

    > Not always. Many C programs run on systems without operating systems.
    > For instance, a lot of systems have ways of forcing data into a cache. C
    > doesn't allow you to do this, you must simply trust the compiler to do a
    > reasonable job for you. Generally this is a good thing, compliers are
    > better than people at most types of cache management.


    In any case, the operating system is (probably) part of the C
    "implementation".

    C99 3.12:

    implementation

    particular set of software, running in a particular translation
    environment under particular control options, that performs
    translation of programs for, and supports execution of functions
    in, a particular execution environment

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
    Keith Thompson, Jul 23, 2006
    #19
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Dhananjay
    Replies:
    1
    Views:
    1,098
    sloan
    Dec 18, 2006
  2. Luke Wu

    Abstraction layer between C and CPU

    Luke Wu, Jan 21, 2005, in forum: C Programming
    Replies:
    30
    Views:
    814
    Chris Torek
    Feb 5, 2005
  3. BJ Dierkes

    Database Abstraction Layer And/Or ORM

    BJ Dierkes, Sep 24, 2007, in forum: Python
    Replies:
    1
    Views:
    311
    Bruno Desthuilliers
    Sep 24, 2007
  4. Ole
    Replies:
    4
    Views:
    172
  5. jis
    Replies:
    11
    Views:
    324
    Tobias Müller
    Jan 30, 2013
Loading...

Share This Page