custom allocation and custom type

Discussion in 'Python' started by castironpi, Aug 4, 2008.

  1. castironpi

    castironpi Guest

    Hi all,

    I am trying to create a custom tuple type. I want it to live in a
    custom memory region, which will be a memory-mapped file. Its
    contents cannot be PyObject*. They have to be offsets into mapped
    memory. GetItem( i ) would return:

    (PyObject *)( t->ob_item[ i ]+ mmap_base_addr );

    In the constructor, each item x would be stored as:

    BYTE* x_offset= x- mmap_base_addr;
    t->ob_item[ 0 ]= x_offset;

    For starters, only ints, strings, and other tuples will be members.
    I'll work out the list type and user-defined classes at a later time.
    Each will be required to be already existing in the mmap; that is, be
    of custom types too.

    I had partial success with strings and ints just memcpy-ing the entire
    PyObject structure. They are easier because they are immutable, and
    don't contain references. That's why tuple is my next step: it's
    immutable, but can contain references.

    I want to back up a step and ask some questions about ints first.

    What worries are there in memcpy-ing a PyIntObject? Would it be
    better to compile my own MmapIntObject? I do Py_INCREF on the object
    I return, to ensure it lives on, but I am getting an occasional crash
    on exit. What did I miss? Do I need to readjust the ob_type pointer
    every time I load the module (and lock for exclusive access)? Does
    Python try to free the memory when it exits?

    For the tuples, is there any way I can reuse the tuple code? The
    constructor uses a custom pool for speed which I'd probably eliminate,
    for the time. Slices would have to be MmapTupleObject instances too.
    Should I also create MmapTupleIterObject?

    PyObject_GC_New and PyObject_GC_NewVar are confusing. I can't follow
    them to the actual call to 'malloc', since they themselves lead around
    Python's custom memory pool. What steps do I need to be sure to take
    in my own?

    I'm having trouble building PerstTupleObject.c (persistent tuple
    object) to try it out. I'm using MinGW on WinXP. I added #include
    "persttupleobject.h" to the top of "persttupleobject.c" already, but
    it's a duplicate otherwise, with the name replaced.

    ---
    C>c:/programs/mingw/bin/g++ persttupleobject.c -c -Ic:/programs/
    python25/include

    persttupleobject.c:650: warning: 'PyPerstTuple_Type' defined locally
    after being
    referenced with dllimport linkage

    C>c:/programs/mingw/bin/g++ -shared persttupleobject.o -o
    persttupleobject.pyd -L
    c:/programs/python25/libs -lpython25

    persttupleobject.o(.text+0xce):persttupleobject.c: undefined reference
    to `_imp_
    _PyPerstTuple_Type'
    persttupleobject.o(.text+0x15c):persttupleobject.c: undefined
    reference to `_PyG
    C_generation0'
    [snip]

    ---
    I get some of the same errors when compiling tupleobject.c from the
    Python-2.5.2 distribution:

    C>c:/programs/mingw/bin/g++ tupleobject.c -c -Ic:/programs/python25/
    include

    tupleobject.c:649: warning: 'PyTuple_Type' defined locally after being
    reference
    d with dllimport linkage

    C>c:/programs/mingw/bin/g++ -shared tupleobject.o -o tupleobject.pyd -
    Lc:/progra
    ms/python25/libs -lpython25

    c:/programs/python25/libs/libpython25.a(dcbbs00622.o)(.text+0x0):
    multiple defin
    ition of `PyTuple_Type'
    tupleobject.o(.data+0x140):tupleobject.c: first defined here
    tupleobject.o(.text+0x15c):tupleobject.c: undefined reference to
    `_PyGC_generati
    on0'
    [snip]

    ---
    Thanks in advance.
     
    castironpi, Aug 4, 2008
    #1
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ken
    Replies:
    24
    Views:
    3,896
    Ben Bacarisse
    Nov 30, 2006
  2. chris
    Replies:
    6
    Views:
    1,007
    chris
    Oct 28, 2005
  3. Frederiek
    Replies:
    1
    Views:
    391
    Thomas Tutone
    Sep 14, 2006
  4. Replies:
    2
    Views:
    374
    Daniel Pitts
    Mar 21, 2007
  5. Bjarke Hammersholt Roune
    Replies:
    14
    Views:
    1,207
    Bjarke Hammersholt Roune
    Mar 6, 2011
Loading...

Share This Page