Core dump while accessing contents from Memory Mapped File

Discussion in 'C Programming' started by nkrisraj@gmail.com, Jan 10, 2006.

  1. Guest

    Hi,

    I have a following structure:
    typedef struct
    {
    RateData rdr;
    int RateID;
    char RateBalance;

    } RateInfo;

    typedef struct
    {
    DayRates *dr;
    WeekRates *wr;

    } RateData;

    First I will create the memoefy for the DayRates, WeekRates structures
    inside the RateInfo structure variable. After that Iam going to copy
    the contents into DayRates, WeekRatesstructures. Once the all the
    structure members inside RateInfo structure variable are initialized, I
    will add that node to a AVL tree. This way, Iam going to create an AVL
    tree by adding RateInfo nodes to it. Once the total tree is contructed,
    I will store this tree as file (rateinfo.map) on the disk.

    In my next programming component, Iam going to load this file
    (rateinfo.map) into memory using mmap() function call. Once I load the
    file into memory using mmap(), Iam able to access the RateID and
    RateBalance members of the RateInfo. But, Iam not able to access the
    DayRates, and WeekRates structure contents. When I try to access the
    members of DayRates , and WeekRates structures, my program is showing
    segmentation fault and Iam getting the core dump.

    Even though I allocated memory for DayRates , and WeekRates structures
    inside RateInfo node while creating the tree, why Iam not able to
    access contents of DayRates , and WeekRates structures when I mmap()
    file into memory?

    Any pointers would be a great help.

    Thanks,
    -Nikhil
    , Jan 10, 2006
    #1
    1. Advertising

  2. In article <>,
    <> wrote:


    >I have a following structure:
    >typedef struct
    >{
    > RateData rdr;
    > int RateID;
    > char RateBalance;
    >} RateInfo;


    >typedef struct
    >{
    > DayRates *dr;
    > WeekRates *wr;
    >} RateData;



    >First I will create the memoefy for the DayRates, WeekRates structures
    > inside the RateInfo structure variable.


    >Once the total tree is contructed,
    >I will store this tree as file (rateinfo.map) on the disk.


    >In my next programming component, Iam going to load this file
    >(rateinfo.map) into memory using mmap() function call. Once I load the
    >file into memory using mmap(), Iam able to access the RateID and
    >RateBalance members of the RateInfo. But, Iam not able to access the
    >DayRates, and WeekRates structure contents.


    It is not clear from what you wrote whether the "next programming
    component" is running in the same process that created the stored tree,
    and it is not clear as to whether you released the memory for the
    tree between the two parts.

    When you write out the tree to a file, you are writing the raw binary
    pointer values (DayRates *)dr and (WeekRates *)wr. Those pointer values
    are to active memory. If you release that memory, either by freeing it
    or overwriting it or by exiting the process, then the referrants of
    the pointers cease to exist. If you then bring back the binary values
    via the mmap() and try to use the stored pointers, you are going to
    be pointing to some random thing. You were lucky in a way to get a
    segment fault, because that made it clear that you were doing something
    wrong; it would have been worse if there hadn't been any fault and
    you ended up using the random data.


    In general, there is no way to store and restore pointers between
    sessions. What people often do instead is to allocate one big
    block of memory to start with, then use their own malloc() and free()
    routines to ensure that memory related to that purpose is taken from
    that block of memory. Then when it is time to write the data out, they
    convert each pointer into an offset from the beginning of the data
    structure. When they want to load the data structure back in again,
    they reverse the process, allocating a block of memory, and converting
    each stored offset into a pointer by adding in the address of the
    new instance of the memory block.


    There is sometimes another option, that is not portable in terms of
    the C standard, but possible if you are willing to restrict execution
    to a particular OS and architecture. In the variant I am thinking of,
    possible in a number of unix-like systems [but not in C in general],
    you can use mmap() to allocate new memory into a fixed virtual address.
    You would then do your own malloc() and free() as per the above
    (or possibly you might find an extension malloc library to do it for you).
    When you finished constructing the tree, you would then save the mmap()
    block to a file. Then to restore the tree into a different process on
    the same kind of system, you would mmap() the file contents into the
    exact same virtual address as was used originally. The pointers would
    then all become valid because the contents and virtual addresses would
    have been restored.

    This option does, of course, assume that your operating system supports
    virtual memory, and supports mmap() as a way of allocating new memory,
    and supports mmap() of a file to a user- specified virtual address, and
    assumes that whatever system the file is restored into has the
    same representation of all the types (including pointers!) as the
    data was created upon...
    --
    All is vanity. -- Ecclesiastes
    Walter Roberson, Jan 10, 2006
    #2
    1. Advertising

  3. Guest

    Hi,

    Actually, I have two components, I mean two different processes. The
    first component creates the tree and writes it to a file. Once it
    writes the tree to a file, the first compoent process exits.

    The second component, will be launched as a seperate process. It takes
    the file created by the first compoent, and does the mmap() it to the
    memory. Once I load the
    file into memory using mmap(), Iam able to access the RateID and
    RateBalance members of the RateInfo. But, Iam not able to access the
    DayRates, and WeekRates structure contents. When I try to access the
    members of DayRates , and WeekRates structures, my program is showing
    segmentation fault and Iam getting the core dump.

    Thanks,
    -Nikhil
    , Jan 10, 2006
    #3
  4. Eric Sosman Guest

    wrote On 01/10/06 12:53,:
    > Hi,
    >
    > Actually, I have two components, I mean two different processes. The
    > first component creates the tree and writes it to a file. Once it
    > writes the tree to a file, the first compoent process exits.
    >
    > The second component, will be launched as a seperate process. It takes
    > the file created by the first compoent, and does the mmap() it to the
    > memory. Once I load the
    > file into memory using mmap(), Iam able to access the RateID and
    > RateBalance members of the RateInfo. But, Iam not able to access the
    > DayRates, and WeekRates structure contents. When I try to access the
    > members of DayRates , and WeekRates structures, my program is showing
    > segmentation fault and Iam getting the core dump.


    Aha! In your initial post you showed a struct
    containing pointers. Did the first process, by any
    chance, write the pointer values themselves to the
    file, rather than the data they pointed to? Those
    pointer values have no meaning in the second process,
    and the pointed-to data has disappeared along with
    everything else that was in the first process' memory.

    I'm guessing, of course, since you haven't provided
    a lot of detail. But if I've guessed correctly, you'll
    need to re-think your file format.

    --
    Eric Sosman, Jan 10, 2006
    #4
  5. In article <>,
    <> wrote:

    >Actually, I have two components, I mean two different processes.


    As -I- just wrote the posting to you a few minutes ago, I know what you
    are talking about... but other people won't. Please take the time to
    quote appropriate context. google groups is giving you the illusion
    that the context is easily visible, but most of the regulars here
    (especially those who answer questions a lot) are -not- using google
    groups. Please read http://cfaj.freeshell.org/google/


    >The first component creates the tree and writes it to a file. Once it
    >writes the tree to a file, the first compoent process exits.


    >The second component, will be launched as a seperate process. It takes
    >the file created by the first compoent, and does the mmap() it to the
    >memory. Once I load the
    >file into memory using mmap(), Iam able to access the RateID and
    >RateBalance members of the RateInfo. But, Iam not able to access the
    >DayRates, and WeekRates structure contents.


    Are you certain of that? I strongly suspect that you -can- access those
    structure contents. The structure contents are *pointers*, and I
    believe that if you check, you will find that the -pointers- can be
    read and have the same value that you stored in the first process.
    I believe you will find that what you cannot access is that which
    is pointed *to* . Which, as I explained in the previous posting, is
    because the pointer values are essentially random garbage in the
    context of the second process. I gave two solutions, one portable and
    the other not, in the previous posting.
    --
    Programming is what happens while you're busy making other plans.
    Walter Roberson, Jan 10, 2006
    #5
  6. In article <dq0t3h$6jl$>,
    Eric Sosman <> wrote:
    > Aha! In your initial post you showed a struct
    >containing pointers. Did the first process, by any
    >chance, write the pointer values themselves to the
    >file, rather than the data they pointed to?


    I'd be surprised if it turned out that he did -not- store the actual
    pointer values and then tried to use the (now-broken) pointers as-is
    in the second process.

    >Those
    >pointer values have no meaning in the second process,
    >and the pointed-to data has disappeared along with
    >everything else that was in the first process' memory.


    With the exception, of course, of possibilities such as having mmap()'d
    back into exactly the same virtual address space. Not generally portable,
    but then mmap() is not generally portable.
    --
    "It is important to remember that when it comes to law, computers
    never make copies, only human beings make copies. Computers are given
    commands, not permission. Only people can be given permission."
    -- Brad Templeton
    Walter Roberson, Jan 10, 2006
    #6
  7. Guest

    Hi,

    In the second process, I can access the pointer addresses exactly, but
    not the pointed values. I mean to say that the addresses of pointers,
    DayRates* and WeekRates* are exactly proper as I can see in the first
    process. But, I can not access the member values of DayRates* and
    WeekRates* structures. When I tried to do that I will get the CORE
    dump.

    Thanks,
    -Nikhil
    , Jan 10, 2006
    #7
  8. In article <>,
    <> wrote:
    >In the second process, I can access the pointer addresses exactly, but
    >not the pointed values. I mean to say that the addresses of pointers,
    >DayRates* and WeekRates* are exactly proper as I can see in the first
    >process. But, I can not access the member values of DayRates* and
    >WeekRates* structures. When I tried to do that I will get the CORE
    >dump.


    You still aren't quoting context :(
    The URL describing how to do so with googlegroups was
    http://cfaj.freeshell.org/google/


    Resupplying context:

    You are writing actual pointer values to a file, exiting that process,
    starting another and reading the pointer values in from the file.
    You happen to be using mmap() for that, but that's not what is causing
    the problem: the problem is that those pointers are no longer valid
    because the stored data is not being loaded back into the exact same
    addresses as it was at when you saved the file.


    To fix this problem, you can use either of the solutions that I
    posted earlier: you can convert the pointers into offsets before you
    write them out, and then convert them back from offsets when you
    read them back in; OR [if your system supports it] you can
    [non-portably] mmap() the contents back to the *exact* same address
    location that the memory had originally... provided that you
    took care to manage memory allocation into a block yourself in the
    first place rather than just using malloc() and free() [which could
    use memory all over your address space.]


    Hmmm, now that I think about it, I imagine that in some cases
    mmap() back to the same virtual location would not work either.
    It is allowable for a "pointer" not to include the complete information
    needed to locate the target memory. For example, a "pointer" might
    internally be a bit-structure that has a segment number reference and
    an offset, or might contain an implicit offset relative to a base
    register. If you were to write that information out and pull it in to
    a different process, then unless -somehow- the segment numbers or
    base registers or whatever also happened to be set to the value
    they had in the other process, then the pointers will fail in the new
    process. If I understand correctly, x86 architectures may use segment +
    offset format. Any architecture that allows access to more physical
    memory than is implied by the native ABI pointer width could encounter
    the same problem.
    --
    I was very young in those days, but I was also rather dim.
    -- Christopher Priest
    Walter Roberson, Jan 10, 2006
    #8
  9. writes:
    > Hi,
    >
    > I have a following structure:
    > typedef struct
    > {
    > RateData rdr;
    > int RateID;
    > char RateBalance;
    >
    > } RateInfo;

    [snip]

    You've posted nearly the same article three times. A lot of people
    have been doing this lately. Apparently groups.google.com has been
    having some problems in the last few days (even more than their
    usual problems; see <http://cfaj.freeshell.org/google/>).

    If you post an article through Google and it doesn't show up
    immediately, don't assume that you need to try again. You may just
    need to wait a while. Posting again won't help; you'll just get two
    copies of the article showing up whenever Google gets around to
    posting.

    --
    Keith Thompson (The_Other_Keith) <http://www.ghoti.net/~kst>
    San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
    We must do something. This is something. Therefore, we must do this.
    Keith Thompson, Jan 10, 2006
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. halfdog
    Replies:
    12
    Views:
    12,418
  2. Replies:
    3
    Views:
    292
    Walter Roberson
    Jan 10, 2006
  3. Replies:
    0
    Views:
    263
  4. Chris Brooks

    memory mapped tar file contents

    Chris Brooks, Nov 13, 2008, in forum: Python
    Replies:
    1
    Views:
    354
    Aaron Brady
    Nov 13, 2008
  5. Frank Seitz

    Core dump with "in memory" file

    Frank Seitz, Mar 5, 2010, in forum: Perl Misc
    Replies:
    4
    Views:
    159
    Peter J. Holzer
    Mar 6, 2010
Loading...

Share This Page