Cpp + Python: static data dynamic initialization in *nix shared lib?

Discussion in 'Python' started by Alf P. Steinbach /Usenet, Jul 9, 2010.

  1. [Cross-posted comp.lang.python and comp.lang.c++]

    I lack experience with shared libraries in *nix and so I need to ask...

    This is about "cppy", some support for writing Python extensions in C++ that I
    just started on (some days ago almost known as "pynis" (not funny after all)).

    For an extension module it seems that Python requires each routine to be defined
    as 'extern "C"'. And although e.g. MSVC is happy to mix 'extern "C"' and C++
    linkage, using a routine declared as 'static' in a class as a C callback,
    formally they're two different kinds, and I seem to recall that /some/ C++
    compiler balks at that kind of mixing unless specially instructed to allow it.
    Perhaps it was the Sun compiler?

    Anyway, to be formally correct I cannot generate the required C routines via
    templating, and I ended up using macros that the user must explicitly invoke,
    like, here the Py doc's first extension module example recoded using cppy,


    ------------------------------------------------------------------
    <code file="spam.cpp">
    #include <progrock/cppx/devsupport/better_experience.h>
    #include <progrock/cppy/Module.h>
    using namespace progrock;

    class Spam: public cppy::Module
    {
    public:
    Spam(): cppy::Module( "spam" )
    {
    setDocString( L"blåbærsyltetøy er blått" );
    }

    PyObject* system( PyObject* args )
    {
    const char *command;
    int sts;

    if( !PyArg_ParseTuple( args, "s", &command ) )
    {
    return NULL;
    }
    sts = ::system( command );
    return Py_BuildValue( "i", sts );
    }
    };

    CPPY_MODULE_CROUTINE( Spam, system, "Execute a shell command" )

    PyMODINIT_FUNC PyInit_spam()
    {
    return cppy::init< Spam >();
    }
    </code>
    ------------------------------------------------------------------


    It works in Windows.

    But here CPPY_MODULE_CROUTINE does three things:

    A Defining the 'extern "C"' routine.
    I cannot think of any problem here.

    B Defining installation data for that routine.
    Possible problem: initializing a static with address of routine?

    C -> Adding that install data record into a linked list!
    Possible problem: are dynamic initialization actions guaranteed
    to be performed in *nix shared library?

    Problem (C) is outside the realm of the C++ standard, since the C++ standard
    doesn't support shared libraries, and I've never actually used *nix shared
    libraries so I don't /know/...

    Is such dynamic initialization guaranteed?

    For completeness, the macro definition (the 0 in there is a list next-pointer):


    <code>
    #define CPPY_MODULE_CROUTINE_DEF( cppClassName, name ) \
    extern "C" \
    static PyObject* cppClassName##_##name( PyObject*, PyObject* args ) \
    { \
    return ::progrock::cppy::module<cppClassName>().name( args ); \
    }

    #define CPPY_MODULE_CROUTINE_INSTALLDATA( cppClassName, name, docString ) \
    static ::progrock::cppy::detail::ModuleRoutineDescriptor \
    cppClassName##_##name##_descriptor = { \
    0, \
    #name, \
    docString, \
    &cppClassName##_##name \
    }; \
    \
    static bool cppClassName##_##name##_descriptor_installed = \
    ::progrock::cppy::detail::addToList< cppClassName >( \
    cppClassName##_##name##_descriptor \
    );

    #define CPPY_MODULE_CROUTINE( cppClassName, name, docString ) \
    CPPY_MODULE_CROUTINE_DEF( cppClassName, name ) \
    CPPY_MODULE_CROUTINE_INSTALLDATA( cppClassName, name, docString )
    </code>


    TIA.,

    - Alf

    --
    blog at <url: http://alfps.wordpress.com>
    Alf P. Steinbach /Usenet, Jul 9, 2010
    #1
    1. Advertising

  2. Alf P. Steinbach /Usenet

    Ian Collins Guest

    Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    On 07/10/10 03:52 AM, Alf P. Steinbach /Usenet wrote:
    > [Cross-posted comp.lang.python and comp.lang.c++]
    >
    > I lack experience with shared libraries in *nix and so I need to ask...
    >
    > This is about "cppy", some support for writing Python extensions in C++
    > that I just started on (some days ago almost known as "pynis" (not funny
    > after all)).
    >
    > For an extension module it seems that Python requires each routine to be
    > defined as 'extern "C"'. And although e.g. MSVC is happy to mix 'extern
    > "C"' and C++ linkage, using a routine declared as 'static' in a class as
    > a C callback, formally they're two different kinds, and I seem to recall
    > that /some/ C++ compiler balks at that kind of mixing unless specially
    > instructed to allow it. Perhaps it was the Sun compiler?


    Yes, it will (correctly) issue a warning.

    As the is a bit OT, contact me directly and we can work through it. I
    have had similar fun and games adding PHP modules!

    --
    Ian Collins
    Ian Collins, Jul 9, 2010
    #2
    1. Advertising

  3. Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    On Fri, Jul 9, 2010 at 5:22 PM, Ian Collins <> wrote:
    > On 07/10/10 03:52 AM, Alf P. Steinbach /Usenet wrote:
    >>
    >> [Cross-posted comp.lang.python and comp.lang.c++]
    >>
    >> I lack experience with shared libraries in *nix and so I need to ask...
    >>
    >> This is about "cppy", some support for writing Python extensions in C++
    >> that I just started on (some days ago almost known as "pynis" (not funny
    >> after all)).
    >>
    >> For an extension module it seems that Python requires each routine to be
    >> defined as 'extern "C"'. And although e.g. MSVC is happy to mix 'extern
    >> "C"' and C++ linkage, using a routine declared as 'static' in a class as
    >> a C callback, formally they're two different kinds, and I seem to recall
    >> that /some/ C++ compiler balks at that kind of mixing unless specially
    >> instructed to allow it. Perhaps it was the Sun compiler?

    >
    > Yes, it will (correctly) issue a warning.
    >
    > As the is a bit OT, contact me directly and we can work through it.  I have
    > had similar fun and games adding PHP modules!


    I'd appreciate it if you'd either leave this on-list or cc me in on this, as
    I'm working through a similar issue.

    Geremy Condra
    geremy condra, Jul 9, 2010
    #3
  4. Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    * Ian Collins, on 09.07.2010 23:22:
    > On 07/10/10 03:52 AM, Alf P. Steinbach /Usenet wrote:
    >> [Cross-posted comp.lang.python and comp.lang.c++]
    >>
    >> I lack experience with shared libraries in *nix and so I need to ask...
    >>
    >> This is about "cppy", some support for writing Python extensions in C++
    >> that I just started on (some days ago almost known as "pynis" (not funny
    >> after all)).
    >>
    >> For an extension module it seems that Python requires each routine to be
    >> defined as 'extern "C"'. And although e.g. MSVC is happy to mix 'extern
    >> "C"' and C++ linkage, using a routine declared as 'static' in a class as
    >> a C callback, formally they're two different kinds, and I seem to recall
    >> that /some/ C++ compiler balks at that kind of mixing unless specially
    >> instructed to allow it. Perhaps it was the Sun compiler?

    >
    > Yes, it will (correctly) issue a warning.
    >
    > As the is a bit OT, contact me directly and we can work through it. I
    > have had similar fun and games adding PHP modules!


    Thanks. I'm mailing you a zip with the code... <g>

    The question, of course, whether it works in *nix.


    Cheers,

    - Alf

    --
    blog at <url: http://alfps.wordpress.com>
    Alf P. Steinbach /Usenet, Jul 9, 2010
    #4
  5. Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    * geremy condra, on 09.07.2010 23:43:
    > On Fri, Jul 9, 2010 at 5:22 PM, Ian Collins<> wrote:
    >> On 07/10/10 03:52 AM, Alf P. Steinbach /Usenet wrote:
    >>>
    >>> [Cross-posted comp.lang.python and comp.lang.c++]
    >>>
    >>> I lack experience with shared libraries in *nix and so I need to ask...
    >>>
    >>> This is about "cppy", some support for writing Python extensions in C++
    >>> that I just started on (some days ago almost known as "pynis" (not funny
    >>> after all)).
    >>>
    >>> For an extension module it seems that Python requires each routine to be
    >>> defined as 'extern "C"'. And although e.g. MSVC is happy to mix 'extern
    >>> "C"' and C++ linkage, using a routine declared as 'static' in a class as
    >>> a C callback, formally they're two different kinds, and I seem to recall
    >>> that /some/ C++ compiler balks at that kind of mixing unless specially
    >>> instructed to allow it. Perhaps it was the Sun compiler?

    >>
    >> Yes, it will (correctly) issue a warning.
    >>
    >> As the is a bit OT, contact me directly and we can work through it. I have
    >> had similar fun and games adding PHP modules!

    >
    > I'd appreciate it if you'd either leave this on-list or cc me in on this, as
    > I'm working through a similar issue.


    Well, we got no further, but I know of three solutions:

    A) Punting: just say that the compiler has to support C++/C function type
    mingling.
    -> Perhaps the practical solution, but formally unsafe.

    B) On the script side of things, delegate all calls to single Mother Of All
    C func downcaller that supplies as extra arg an id of the C++ function.
    -> Micro-level inefficient but easy to use and formally correct.

    C) Let the user define the C linkage function wrappers via macros.
    -> Efficient and formally correct but exposes ugly macro names.

    I chose (C).

    I believe Boost's Python binding uses (A), or perhaps (B).


    Cheers,

    - Alf

    PS: You (the reader) may be wondering, why why why Yet Another Python/C++
    binding? Well, because I had this great name for it, "pyni", unfortunately
    already in use. But cppy is very different from Boost: Boost is large, cppy is
    tiny; Boost has as main goal to expose arbitrary C++ code to Python, automating
    argument conversion etc., while with cppy your Python design is exposed to C++
    with no enforced arg conversions and such; Boost relies on canned magic,
    difficult to subvert when it doesn't do what you want, while with cppy you are
    (or, so far, I am) in control; and I suspect that the Boost Python binding,
    relying on dynamic registries and stuff, is not all that efficient, while cppy
    is as efficient as using the Python C API to create an extension. And besides,
    cppy supports national characters in doc strings etc. And I'm Norwegian. So. :)

    --
    blog at <url: http://alfps.wordpress.com>
    Alf P. Steinbach /Usenet, Jul 13, 2010
    #5
  6. Alf P. Steinbach /Usenet

    Jonathan Lee Guest

    Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    > Problem (C) is outside the realm of the C++ standard, since the C++ standard
    > doesn't support shared libraries, and I've never actually used *nix shared
    > libraries so I don't /know/...
    >
    > Is such dynamic initialization guaranteed?
    >


    Not guaranteed, though I think there's a combination of dlopen options
    and gcc command line parameters that invoke this behavior. See the
    second page of

    http://www.linuxjournal.com/article/3687

    about auto-registration.

    Personally, though, it never worked for me :/

    --Jonathan
    Jonathan Lee, Jul 13, 2010
    #6
  7. Alf P. Steinbach /Usenet

    Robert Kern Guest

    Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    On 7/13/10 2:34 AM, Alf P. Steinbach /Usenet wrote:

    > PS: You (the reader) may be wondering, why why why Yet Another Python/C++
    > binding? Well, because I had this great name for it, "pyni", unfortunately
    > already in use. But cppy is very different from Boost: Boost is large, cppy is
    > tiny; Boost has as main goal to expose arbitrary C++ code to Python, automating
    > argument conversion etc., while with cppy your Python design is exposed to C++
    > with no enforced arg conversions and such; Boost relies on canned magic,
    > difficult to subvert when it doesn't do what you want, while with cppy you are
    > (or, so far, I am) in control; and I suspect that the Boost Python binding,
    > relying on dynamic registries and stuff, is not all that efficient, while cppy
    > is as efficient as using the Python C API to create an extension. And besides,
    > cppy supports national characters in doc strings etc. And I'm Norwegian. So. :)


    Note that Boost is not the only C++ binding out there. You may want to take a
    look at the old SCXX library, which appears to be similar in intent:

    http://davidf.sjsoft.com/mirrors/mcmillan-inc/scxx.html

    matplotlib uses it heavily, and their included copy may include some more recent
    bugfixes and enhancements:

    http://matplotlib.sourceforge.net/

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
    Robert Kern, Jul 13, 2010
    #7
  8. Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    * Robert Kern, on 13.07.2010 17:16:
    > On 7/13/10 2:34 AM, Alf P. Steinbach /Usenet wrote:
    >
    >> PS: You (the reader) may be wondering, why why why Yet Another Python/C++
    >> binding? Well, because I had this great name for it, "pyni", unfortunately
    >> already in use. But cppy is very different from Boost: Boost is large, cppy is
    >> tiny; Boost has as main goal to expose arbitrary C++ code to Python, automating
    >> argument conversion etc., while with cppy your Python design is exposed to C++
    >> with no enforced arg conversions and such; Boost relies on canned magic,
    >> difficult to subvert when it doesn't do what you want, while with cppy you are
    >> (or, so far, I am) in control; and I suspect that the Boost Python binding,
    >> relying on dynamic registries and stuff, is not all that efficient, while cppy
    >> is as efficient as using the Python C API to create an extension. And besides,
    >> cppy supports national characters in doc strings etc. And I'm Norwegian. So. :)

    >
    > Note that Boost is not the only C++ binding out there. You may want to
    > take a look at the old SCXX library, which appears to be similar in intent:
    >
    > http://davidf.sjsoft.com/mirrors/mcmillan-inc/scxx.html
    >
    > matplotlib uses it heavily, and their included copy may include some
    > more recent bugfixes and enhancements:
    >
    > http://matplotlib.sourceforge.net/


    Thanks! It seems that SCXX does those things that I've been planning to do but
    haven't got around to (wrapping standard Python types), while what it doesn't do
    (abstracting away all those tables etc. and mapping Python calls to C++ calls)
    is what I've been working on. Which could be a Very Nice combination except that
    I'm assuming Py3, while SCXX seems to be Py2 only. :-(


    Cheers,

    - Alf

    --
    blog at <url: http://alfps.wordpress.com>
    Alf P. Steinbach /Usenet, Jul 13, 2010
    #8
  9. Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    * Jonathan Lee, on 13.07.2010 16:41:
    >> Problem (C) is outside the realm of the C++ standard, since the C++ standard
    >> doesn't support shared libraries, and I've never actually used *nix shared
    >> libraries so I don't /know/...
    >>
    >> Is such dynamic initialization guaranteed?
    >>

    >
    > Not guaranteed, though I think there's a combination of dlopen options
    > and gcc command line parameters that invoke this behavior. See the
    > second page of
    >
    > http://www.linuxjournal.com/article/3687
    >
    > about auto-registration.
    >
    > Personally, though, it never worked for me :/


    Ah, well. :-( Thanks for the info! OK, I'll just have to replace the
    auto-registration with some C++ magic. For which I think I'll simply /require/
    that the compiler supports mixing of C and C++ linkage, that is, that ...


    <code language="Not quite standard C++!">
    #include <iostream>

    extern "C"
    {
    typedef int (*Callback)( int );
    }

    void foo( Callback f ) { std::cout << "foo!" << f( 42 ) << std::endl; }

    int a( int ) { return 1; }
    extern "C" int b( int ) { return 2; }

    int main()
    {
    foo( a ); // Unholy Mix of C++ and C linkage, formally not OK.
    foo( b ); // Should be OK with any compiler.
    }
    </code>


    .... compiles, and works.


    Cheers, & thanks,

    - Alf

    --
    blog at <url: http://alfps.wordpress.com>
    Alf P. Steinbach /Usenet, Jul 13, 2010
    #9
  10. Alf P. Steinbach /Usenet

    sturlamolden Guest

    Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    On 9 Jul, 17:52, "Alf P. Steinbach /Usenet" <alf.p.steinbach
    > wrote:

    > For an extension module it seems that Python requires each routine to be defined
    > as 'extern "C"'.


    That is strange. PyMethodDef is just a jump table. So why should
    'extern "C"' matter?

    Good luck on re-inventing the wheel (you've probably heared about
    Swig, SIP, Boost.Python, PyCXX, scipy.weave and Cython...)
    sturlamolden, Jul 13, 2010
    #10
  11. Alf P. Steinbach /Usenet

    sturlamolden Guest

    Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    On 13 Jul, 21:39, "Alf P. Steinbach /Usenet" <alf.p.steinbach
    > wrote:

    > Thanks! It seems that SCXX does those things that I've been planning to do but
    > haven't got around to (wrapping standard Python types), while what it doesn't do
    > (abstracting away all those tables etc. and mapping Python calls to C++ calls)
    > is what I've been working on. Which could be a Very Nice combination except that
    > I'm assuming Py3, while SCXX seems to be Py2 only. :-(


    I'd suggest PyCXX instead. http://cxx.sourceforge.net

    SCXX is a tiny wrapper mostly used in scipy.weave to inline C++ in
    Python.
    sturlamolden, Jul 13, 2010
    #11
  12. Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    * sturlamolden, on 13.07.2010 22:03:
    > On 9 Jul, 17:52, "Alf P. Steinbach /Usenet"<alf.p.steinbach
    > > wrote:
    >
    >> For an extension module it seems that Python requires each routine to be defined
    >> as 'extern "C"'.

    >
    > That is strange. PyMethodDef is just a jump table. So why should
    > 'extern "C"' matter?


    Formally because they're incompatible function pointer types.

    C++98 standard §7.5/1: "Two function types with different language linkages are
    distinct types even if they are otherwise identical". Add to that §7.5/4 "A
    linkage-specification shall occur only in namespace scope". And add to that
    §14-4 "A template, a template explicit specialization, or a class-template
    partial specialization shall not have C linkage." This means that formally
    correct code that generates callbacks by templating, is ruled out.

    In practice, 'extern "C"' matters for the jump tables because for those few
    compilers if any where it really matters (not just the compiler emitting a
    warning like reportedly Sun CC does), different linkage can imply different
    machine code level calling convention. For example, who's responsible for
    cleaning up the stack, the order in which arguments are pushed or which
    registers they're passed in, and so forth. Ignoring such matters your code gets
    into la-la land pretty fast, but, it's a different matter when one /understands/
    this and places a requirement on the compiler.


    > Good luck on re-inventing the wheel (you've probably heared about
    > Swig, SIP, Boost.Python, PyCXX, scipy.weave and Cython...)


    Yes, I know Boost.Python in more detail and I've heard of all the rest except
    SIP, but then regarding SIP I really don't like QT (QT makes eminent sense in
    the context of Python, they're both essentially dynamically typed, but that
    means QT is not very nice as C++, plus there is the ugly preprocessor).

    And as you'd guess if you were not in silly ignoramus assertion-mode, I'm not
    reinventing the wheel.


    Cheers & hth.,

    - Alf

    --
    blog at <url: http://alfps.wordpress.com>
    Alf P. Steinbach /Usenet, Jul 13, 2010
    #12
  13. Alf P. Steinbach /Usenet

    Dilip Guest

    Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    On Jul 13, 2:34 am, "Alf P. Steinbach /Usenet" <alf.p.steinbach
    > wrote:
    > Well, we got no further, but I know of three solutions:
    >
    >    A) Punting: just say that the compiler has to support C++/C function type
    >       mingling.
    >       -> Perhaps the practical solution, but formally unsafe.
    >
    >    B) On the script side of things, delegate all calls to single Mother Of All
    >       C func downcaller that supplies as extra arg an id of the C++ function.
    >       -> Micro-level inefficient but easy to use and formally correct.
    >
    >    C) Let the user define the C linkage function wrappers via macros.
    >       -> Efficient and formally correct but exposes ugly macro names.
    >
    > I chose (C).


    Alf

    This may or may not be what you are looking for but the middleware Ice
    provides language mapping to enable Python to call into the Ice
    libraries which are basically written in C++. You can take a look at
    this: http://www.zeroc.com/icepy.html

    However that page may not be very descriptive. The codebase, though,
    is freely downloadable. You can take a look at it if that will help
    although you need to wade around a little bit to figure out what is
    where.
    Dilip, Jul 13, 2010
    #13
  14. Alf P. Steinbach /Usenet

    sturlamolden Guest

    Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    On 13 Jul, 22:35, "Alf P. Steinbach /Usenet" <alf.p.steinbach
    > wrote:

    > In practice, 'extern "C"' matters for the jump tables because for those few
    > compilers if any where it really matters (not just the compiler emitting a
    > warning like reportedly Sun CC does), different linkage can imply different
    > machine code level calling convention.


    I see. Just stick to MSVC and GNU and that never happens, just do a C
    style cast.

    > Yes, I know Boost.Python in more detail and I've heard of all the rest except
    > SIP, but then regarding SIP I really don't like QT


    You don't have to use Qt to use SIP. It's just a tool to automatically
    wrap C++ for Python (like Swig, except designed for C++).


    > And as you'd guess if you were not in silly ignoramus assertion-mode, I'm not
    > reinventing the wheel.


    It seems you are re-inventing PyCXX (or to a lesser extent
    Boost.Python).
    sturlamolden, Jul 13, 2010
    #14
  15. Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    * sturlamolden, on 13.07.2010 22:06:
    > On 13 Jul, 21:39, "Alf P. Steinbach /Usenet"<alf.p.steinbach
    > > wrote:
    >
    >> Thanks! It seems that SCXX does those things that I've been planning to do but
    >> haven't got around to (wrapping standard Python types), while what it doesn't do
    >> (abstracting away all those tables etc. and mapping Python calls to C++ calls)
    >> is what I've been working on. Which could be a Very Nice combination except that
    >> I'm assuming Py3, while SCXX seems to be Py2 only. :-(

    >
    > I'd suggest PyCXX instead. http://cxx.sourceforge.net
    >
    > SCXX is a tiny wrapper mostly used in scipy.weave to inline C++ in
    > Python.


    Thanks. I looked up your URL, and PyCXX design goals seem to be much like what
    I'm doing, but different in some crucial ways. It's probably great, but it's not
    to my taste.

    E.g. the PyCXX String class interface:


    explicit String( PyObject *pyob, bool owned = false )
    String( const Object &ob )
    String()
    String( const char *latin1 )
    String( const char *latin1, Py_ssize_t size )
    String( const std::string &latin1 )
    String( const std::string &v, const char *encoding, const char *error=NULL )
    String( const char *s, const char *encoding, const char *error=NULL )
    String( const char *s, Py_ssize_t len, const char *encoding, const char
    *error=NULL )
    String & operator=( const Object &o )
    String & operator=( PyObject *p )
    String & operator=( const unicodestring &v )
    size_type size() const
    size_type capacity() const
    unicodestring as_unicodestring() const
    std::string operator std::string() const
    String encode( const char *encoding, const char *error="strict" )
    std::string as_std_string( const char *encoding=NULL, const char
    *error="strict" ) const


    In C++ the only way to portably specify a string literal with national
    characters, is as a wide string literal. Otherwise the result depends on the
    source code encoding. Yet PyCXX's String does not support wchar_t.

    Also, things like the 'owned' option is just asking for trouble.

    I chose this example because a Python string wrapper is the only object wrapper
    (apart from a general PyPtr) that I've implemented so far. The PyCXX string
    wrapper fails the design criterions of general usability (including in
    particular lack of wide char support) and safety (in particular the 'owned'
    option). And it's underdocumented, like, what encoding does the operator
    std::string() produce?

    The details don't matter though. I'm sure that PyCXX is Very Nice for its
    purpose, as is e.g. Boost.Python (which is Very Very Nice for its purpose). :)


    Cheers, & thanks for the link,

    - Alf

    --
    blog at <url: http://alfps.wordpress.com>
    Alf P. Steinbach /Usenet, Jul 13, 2010
    #15
  16. Alf P. Steinbach /Usenet

    sturlamolden Guest

    Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    On 13 Jul, 22:35, "Alf P. Steinbach /Usenet" <alf.p.steinbach
    > wrote:

    > Yes, I know Boost.Python in more detail and I've heard of all the rest except
    > SIP,



    In my opinion, SIP is the easiest way of integrating C++ and Python.
    Just ignore the PyQt stuff.

    http://www.riverbankcomputing.co.uk/static/Docs/sip4/using.html#a-simple-c-example
    http://www.riverbankcomputing.co.uk/software/sip/intro

    (SIP 4 can also expose C libraries to Python, not just C++.)
    sturlamolden, Jul 13, 2010
    #16
  17. "Alf P. Steinbach /Usenet" <> writes:

    > Also, things like the 'owned' option is just asking for trouble.


    Isn't owned=true (or equivalent) a necessity when initializing from a
    PyObject* returned by a function declared to return a "new reference"?

    How does your API deal with the distinction between new and borrowed
    references?
    Hrvoje Niksic, Jul 14, 2010
    #17
  18. Re: Cpp + Python: static data dynamic initialization in *nix sharedlib?

    * Hrvoje Niksic, on 14.07.2010 10:17:
    > "Alf P. Steinbach /Usenet"<> writes:
    >
    >> Also, things like the 'owned' option is just asking for trouble.

    >
    > Isn't owned=true (or equivalent) a necessity when initializing from a
    > PyObject* returned by a function declared to return a "new reference"?


    No, not quite.

    Consider this signature (PyCXX):

    String( PyObject* pyob, bool owned = false )

    versus this (cppy):

    PyString( PyPtr object )

    With the first signature, every time you construct a String you have to remember
    to (explicitly or implicitly) set the bool flag correctly depending on where the
    actual argument value comes from.

    With the second signature the /type/ of the actual argument decides,
    automatically, so there's much less room for messing things up.

    With the second signature, if the actual argument is a raw PyObject* pointer
    then it's not owned, and the PyPtr formal argument doesn't increment the
    reference count but just takes ownership. If the actual argument, on the other
    hand, is a PyPtr, then the object is owned by the collection of PyPtr instances
    that refer to the object, and the new formal argument PyPtr instance then
    increments the reference count. In passing, if it should happen that the Python
    community uses the word 'owned' in the opposite sense of what's conventional in
    C++, then I guess & hope you'll figure out what I mean from this description.

    PyPtr currently looks like this (complete code):


    <code file="PyPtr.h">
    // progrock.cppy -- "C++ plus Python"
    // A simple C++ framework for writing Python 3.x extensions.
    //
    // Copyright (c) Alf P. Steinbach, 2010.

    #ifndef CPPY_PYPTR_H
    #define CPPY_PYPTR_H
    #include <progrock/cppx/devsupport/better_experience.h>


    //----------------------------------------- Dependencies:

    #include <Python.h>
    #include <assert.h>
    #include <algorithm>



    //----------------------------------------- Interface:

    namespace progrock{ namespace cppy {
    using namespace cppx;

    enum DoAddRef {};

    class PyPtr
    {
    private:
    PyObject* p_;

    public:
    typedef cppy::DoAddRef DoAddRef;

    PyPtr( PyObject* p = 0 ): p_( p ) {}

    PyPtr( PyObject* p, DoAddRef ): p_( p )
    {
    assert( p != 0 ); Py_INCREF( p_ );
    }

    PyPtr( PyPtr const& other ): p_( other.p_ ) { Py_XINCREF( p_ ); }

    ~PyPtr() { Py_XDECREF( p_ ); }

    void swapWith( PyPtr& other ) { std::swap( p_, other.p_ ); }
    PyPtr& operator=( PyPtr other ) { swapWith( other ); return *this; }

    PyObject* get() const { return p_; }

    PyObject* release()
    {
    PyObject* const result = p_;
    p_ = 0;
    return result;
    }
    };

    inline PyObject* withAddedRef( PyObject* p )
    {
    Py_INCREF( p );
    return p;
    }

    inline PyObject* pyNoneRef()
    {
    return withAddedRef( Py_None );
    }
    } } // namespace progrock::cppy


    #endif
    </code>


    As you can see at the end there, there is 'withAddedRef' and 'pyNoneRef', and
    for that matter the 'DoAddRef' and the 'release()' method, for the cases where
    PyPtr default handling doesn't quite cut it... :)


    > How does your API deal with the distinction between new and borrowed
    > references?


    See above.

    There are some cases where it must be dealt with, but the main idea is to encode
    that into /types/, instead of as context-dependent figure-it-out.

    That's also the main idea of C++ as compared to C, to encode much more into
    types, which helps both for compile time error detection and for automating
    things (like above), which is Very Nice, but which also can make for enormous
    waste of time trying to make the darned language do what you want it to do! :)


    Cheers & hth.,

    - Alf


    Disclaimer: it's late in the day/night for me, so the above explanation may have
    big logic holes, mispelings and so on, but I hope it gives the idea.

    --
    blog at <url: http://alfps.wordpress.com>
    Alf P. Steinbach /Usenet, Jul 14, 2010
    #18
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. James Yong
    Replies:
    0
    Views:
    528
    James Yong
    Sep 12, 2005
  2. Paras Sharma

    CPP with dynamic lib

    Paras Sharma, Jul 29, 2003, in forum: C++
    Replies:
    1
    Views:
    334
    Jack Klein
    Jul 29, 2003
  3. Lorenzo Bettini

    Re: cpp to html (*nix preferred)

    Lorenzo Bettini, Jul 29, 2003, in forum: C++
    Replies:
    0
    Views:
    336
    Lorenzo Bettini
    Jul 29, 2003
  4. est
    Replies:
    1
    Views:
    585
    Diez B. Roggisch
    Feb 16, 2008
  5. Alf P. Steinbach /Usenet
    Replies:
    10
    Views:
    587
    sturlamolden
    Jul 13, 2010
Loading...

Share This Page