Anything new on run-time libraries loading?

Discussion in 'C++' started by Waldek M., Oct 31, 2011.

  1. Waldek M.

    Waldek M. Guest

    Hello,

    The standard way for loading libraries on runtime (in this case, for
    Linux/Unix systems) is dlsym/dlopen & friends.
    The catch is, libraries written with C++ need to have some
    kind of "C" interface (hence not mangled) that would allow access to C++
    objects, at least supporting creation and destruction, like here:
    http://tldp.org/HOWTO/C -dlopen/thesolution.html

    And my question: does the new C++ standard address this issue somehow?
    Or, is there any more comfortable (still standard compliant) way to achieve
    the same goal?

    Best regards,
    Waldek
     
    Waldek M., Oct 31, 2011
    #1
    1. Advertising

  2. Waldek M.

    Goran Guest

    On Oct 31, 6:54 pm, "Waldek M." <> wrote:
    > Hello,
    >
    > The standard way for loading libraries on runtime (in this case, for
    > Linux/Unix systems) is dlsym/dlopen & friends.
    > The catch is, libraries written with C++ need to have some
    > kind of "C" interface (hence not mangled) that would allow access to C++
    > objects, at least supporting creation and destruction, like here:http://tldp.org/HOWTO/C -dlopen/thesolution.html
    >
    > And my question: does the new C++ standard address this issue somehow?
    > Or, is there any more comfortable (still standard compliant) way to achieve
    > the same goal?


    I think both C and C++ languages have no notion whatsoever of
    libraries, *.so or *.lib, including latest C++ spec.

    I personally think they should never get that. C-level
    "interoperability" on library level works primarily because C is so
    simple. He who needs something more than that, C, should reach for
    some integration tech that defines needed binary-level compatibility.
    I find it amazing that virtually no such tech, that is cross-platform
    and in wide use, exist today. There's COM on Windows and that's about
    it. Pffft...

    Goran.
     
    Goran, Oct 31, 2011
    #2
    1. Advertising

  3. Waldek M.

    Richard Guest

    [Please do not mail me a copy of your followup]

    Goran <> spake the secret code
    <> thusly:

    >[...] There's COM on Windows and that's about it.


    I believe the unix equivalent is CORBA.
    --
    "The Direct3D Graphics Pipeline" -- DirectX 9 version available for download
    <http://legalizeadulthood.wordpress.com/the-direct3d-graphics-pipeline/>

    Legalize Adulthood! <http://legalizeadulthood.wordpress.com>
     
    Richard, Nov 1, 2011
    #3
  4. Waldek M.

    Ian Collins Guest

    On 11/ 1/11 06:28 PM, Richard wrote:
    > [Please do not mail me a copy of your followup]
    >
    > Goran<> spake the secret code
    > <> thusly:
    >
    >> [...] There's COM on Windows and that's about it.

    >
    > I believe the unix equivalent is CORBA.


    s/unix/portable/

    --
    Ian Collins
     
    Ian Collins, Nov 1, 2011
    #4
  5. Waldek M.

    Waldek M. Guest

    On Mon, 31 Oct 2011 12:34:47 -0700 (PDT), Goran wrote:
    > I think both C and C++ languages have no notion whatsoever of
    > libraries, *.so or *.lib, including latest C++ spec.


    It looks so, yes.

    > I personally think they should never get that. C-level
    > "interoperability" on library level works primarily because C is so
    > simple.

    For instance, Python is a little bit more complex and yet it tries
    to address inter-operability. Of course, C++ and P. are completely
    different beasts and I'm far from trying any flaming whatsoever.


    > He who needs something more than that, C, should reach for
    > some integration tech that defines needed binary-level compatibility.
    > I find it amazing that virtually no such tech, that is cross-platform
    > and in wide use, exist today.


    For C++, due to non-standarized name mangling, it's not a trivial thing.
    That's why I hoped something has changed recently. Seems like it didn't.
    Oh, well... :)

    Best regards,
    Waldek
     
    Waldek M., Nov 1, 2011
    #5
  6. Waldek M.

    Edek Guest

    On 10/31/2011 08:34 PM, Goran wrote:
    > On Oct 31, 6:54 pm, "Waldek M."<> wrote:
    >> Hello,
    >>
    >> The standard way for loading libraries on runtime (in this case, for
    >> Linux/Unix systems) is dlsym/dlopen& friends.
    >> The catch is, libraries written with C++ need to have some
    >> kind of "C" interface (hence not mangled) that would allow access to C++
    >> objects, at least supporting creation and destruction, like here:http://tldp.org/HOWTO/C -dlopen/thesolution.html
    >>
    >> And my question: does the new C++ standard address this issue somehow?
    >> Or, is there any more comfortable (still standard compliant) way to achieve
    >> the same goal?

    >
    > I think both C and C++ languages have no notion whatsoever of
    > libraries, *.so or *.lib, including latest C++ spec.


    Why would they? If they would, they would need to include llvm IR
    binaries too (they are for example for JIT, still being c++).

    >
    > I personally think they should never get that. C-level
    > "interoperability" on library level works primarily because C is so
    > simple. He who needs something more than that, C, should reach for
    > some integration tech that defines needed binary-level compatibility.
    > I find it amazing that virtually no such tech, that is cross-platform
    > and in wide use, exist today. There's COM on Windows and that's about
    > it. Pffft...


    Speaking of dlsym/dlopen, there is binary compatibility. While I don't
    know all platforms, compilers often agree to the same ABI, so it
    is absolutely no problem to share data and call the code. All you
    need is a little wrapper for _dynamically_ loaded libs, which are
    plugins or whatever and this wrapper is not needed for every
    interaction, just for one (or a couple) entry points used
    as an object factory. So except for the factory stuff, not much
    more is needed - an API of course, but it does not require
    any C linkage, it can be just regular C++.

    The whole point of that is not binary compatibility, but
    the fact that in C++ one cannot call methods (constructors are
    also callable) of unknown at compilation time name and arguments.

    The possible problem with new does exist, but some things
    may be decided by the policy of plugin integration, which
    is needed anyway. There are many applications with plugins,
    none of them uses CORBA or anything similar.

    Edek
     
    Edek, Nov 1, 2011
    #6
  7. Waldek M.

    Dombo Guest

    Op 01-Nov-11 6:28, Richard schreef:
    > [Please do not mail me a copy of your followup]
    >
    > Goran<> spake the secret code
    > <> thusly:
    >
    >> [...] There's COM on Windows and that's about it.

    >
    > I believe the unix equivalent is CORBA.


    More like the cross-platform equivalent; CORBA implantations are
    available for other platforms as well (including Windows). There also
    many other middleware solutions.
     
    Dombo, Nov 1, 2011
    #7
  8. Waldek M.

    Jorgen Grahn Guest

    On Tue, 2011-11-01, Dombo wrote:
    > Op 01-Nov-11 6:28, Richard schreef:
    >> [Please do not mail me a copy of your followup]
    >>
    >> Goran<> spake the secret code
    >> <> thusly:
    >>
    >>> [...] There's COM on Windows and that's about it.

    >>
    >> I believe the unix equivalent is CORBA.

    >
    > More like the cross-platform equivalent; CORBA implantations are
    > available for other platforms as well (including Windows). There also
    > many other middleware solutions.


    Although at that point, you might find yourself saying "maybe a plain
    C interface to the loadable part isn't such a bad idea after all" ...

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
     
    Jorgen Grahn, Nov 2, 2011
    #8
  9. Waldek M.

    Edek Guest

    On 11/01/2011 10:34 PM, Paavo Helde wrote:
    > "Waldek M."<> wrote in
    > news:2u8yuf1f8xnx$:
    >
    >> Hello,
    >>
    >> The standard way for loading libraries on runtime (in this case, for
    >> Linux/Unix systems) is dlsym/dlopen& friends.
    >> The catch is, libraries written with C++ need to have some
    >> kind of "C" interface (hence not mangled) that would allow access to
    >> C++ objects, at least supporting creation and destruction, like here:
    >> http://tldp.org/HOWTO/C -dlopen/thesolution.html
    >>
    >> And my question: does the new C++ standard address this issue somehow?
    >> Or, is there any more comfortable (still standard compliant) way to
    >> achieve the same goal?

    >
    > Adding a couple of extern "C" is as comfortable as you can get. Note that
    > the function marked extern "C" can be otherwise fully C++ in both the
    > definition and signature. There might be some complications, e.g.
    > throwing exceptions across the extern "C" border is not guaranteed to
    > work IIRC.


    That is in the scope of factory methods, all other calls can be C++
    (see the triangle example in the mentioned FAQ)

    >
    > If you want to pass C++ objects, both libraries must be binary compatible
    > anyway, so in principle nothing prevents you sneaking out the mangled
    > name by the nm tool and using that in your dlsym() calls. However, this
    > is much more fragile than using extern "C" (there were times when gcc
    > changed its name mangling rules about every minor release, for example).


    Why would you use nm? Why not make an extern C factory method returning
    an object with virtual methods you can call (same example)?

    Edek
     
    Edek, Nov 2, 2011
    #9
  10. Waldek M.

    Edek Guest

    On 11/02/2011 07:08 AM, Jorgen Grahn wrote:
    > On Tue, 2011-11-01, Dombo wrote:
    >> Op 01-Nov-11 6:28, Richard schreef:
    >>> [Please do not mail me a copy of your followup]
    >>>
    >>> Goran<> spake the secret code
    >>> <> thusly:
    >>>
    >>>> [...] There's COM on Windows and that's about it.
    >>>
    >>> I believe the unix equivalent is CORBA.

    >>
    >> More like the cross-platform equivalent; CORBA implantations are
    >> available for other platforms as well (including Windows). There also
    >> many other middleware solutions.

    >
    > Although at that point, you might find yourself saying "maybe a plain
    > C interface to the loadable part isn't such a bad idea after all" ...


    Either I am missing something huge, or what are you talking about?

    You can have a C++ interface (see the FAQ in OP, the triangle class
    example)

    Edek
     
    Edek, Nov 2, 2011
    #10
  11. Waldek M.

    Dombo Guest

    Op 02-Nov-11 10:03, Edek schreef:
    > On 11/02/2011 07:08 AM, Jorgen Grahn wrote:
    >> On Tue, 2011-11-01, Dombo wrote:
    >>> Op 01-Nov-11 6:28, Richard schreef:
    >>>> [Please do not mail me a copy of your followup]
    >>>>
    >>>> Goran<> spake the secret code
    >>>> <>
    >>>> thusly:
    >>>>
    >>>>> [...] There's COM on Windows and that's about it.
    >>>>
    >>>> I believe the unix equivalent is CORBA.
    >>>
    >>> More like the cross-platform equivalent; CORBA implantations are
    >>> available for other platforms as well (including Windows). There also
    >>> many other middleware solutions.

    >>
    >> Although at that point, you might find yourself saying "maybe a plain
    >> C interface to the loadable part isn't such a bad idea after all" ...

    >
    > Either I am missing something huge, or what are you talking about?
    >
    > You can have a C++ interface (see the FAQ in OP, the triangle class
    > example)


    CORBA is rather complex, you have to be a bit of masochist to enjoy
    dealing with this stuff.
     
    Dombo, Nov 2, 2011
    #11
  12. Waldek M.

    Jorgen Grahn Guest

    On Wed, 2011-11-02, Edek wrote:
    > On 11/02/2011 07:08 AM, Jorgen Grahn wrote:
    >> On Tue, 2011-11-01, Dombo wrote:
    >>> Op 01-Nov-11 6:28, Richard schreef:
    >>>> [Please do not mail me a copy of your followup]
    >>>>
    >>>> Goran<> spake the secret code
    >>>> <> thusly:
    >>>>
    >>>>> [...] There's COM on Windows and that's about it.
    >>>>
    >>>> I believe the unix equivalent is CORBA.
    >>>
    >>> More like the cross-platform equivalent; CORBA implantations are
    >>> available for other platforms as well (including Windows). There also
    >>> many other middleware solutions.

    >>
    >> Although at that point, you might find yourself saying "maybe a plain
    >> C interface to the loadable part isn't such a bad idea after all" ...

    >
    > Either I am missing something huge, or what are you talking about?


    I'm saying that if your problem is the one in the original posting
    (calling code in dlopen()ed libraries) you'd probably rather stick to
    a C interface than go CORBA, or XMLsomething, or ...

    (Perhaps the topic shifted without me noticing it, because I can't see
    how CORBA solves that at all.)

    > You can have a C++ interface (see the FAQ in OP, the triangle class
    > example)


    I missed that part -- but at a quick glance it doesn't seem much more
    useful than a C interface.

    /Jorgen

    --
    // Jorgen Grahn <grahn@ Oo o. . .
    \X/ snipabacken.se> O o .
     
    Jorgen Grahn, Nov 3, 2011
    #12
  13. Waldek M.

    Edek Guest

    On 11/03/2011 07:26 PM, Jorgen Grahn wrote:
    > On Wed, 2011-11-02, Edek wrote:
    >> On 11/02/2011 07:08 AM, Jorgen Grahn wrote:
    >>> On Tue, 2011-11-01, Dombo wrote:
    >>>> Op 01-Nov-11 6:28, Richard schreef:
    >>>>> [Please do not mail me a copy of your followup]
    >>>>>
    >>>>> Goran<> spake the secret code
    >>>>> <> thusly:
    >>>>>
    >>>>>> [...] There's COM on Windows and that's about it.
    >>>>>
    >>>>> I believe the unix equivalent is CORBA.
    >>>>
    >>>> More like the cross-platform equivalent; CORBA implantations are
    >>>> available for other platforms as well (including Windows). There also
    >>>> many other middleware solutions.
    >>>
    >>> Although at that point, you might find yourself saying "maybe a plain
    >>> C interface to the loadable part isn't such a bad idea after all" ...

    >>
    >> Either I am missing something huge, or what are you talking about?

    >
    > I'm saying that if your problem is the one in the original posting
    > (calling code in dlopen()ed libraries) you'd probably rather stick to
    > a C interface than go CORBA, or XMLsomething, or ...


    I misaddressed my post, I think Goran started talking about
    COM as if dlopen limited anybody to C:

    > He who needs something more than that, C, should reach for
    > some integration tech that defines needed binary-level compatibility.
    > I find it amazing that virtually no such tech, that is cross-platform
    > and in wide use, exist today. There's COM on Windows and that's about
    > it.


    So maybe LoadLibrary on Windows limits usage to C and one needs COM,
    I don't know if that is true or not. Yeah, platforms do differ.

    >
    > (Perhaps the topic shifted without me noticing it, because I can't see
    > how CORBA solves that at all.)


    CORBA has a lot of features like load balancing, entities and so on;
    using CORBA for a plugin framework is 'a bit of' an overkill. I see
    no point in using CORBA on a single machine at all, I see the
    major downside of using anything that is so heavy for simple
    method calls, mainly because of PITA for developers.

    >
    >> You can have a C++ interface (see the FAQ in OP, the triangle class
    >> example)

    >
    > I missed that part -- but at a quick glance it doesn't seem much more
    > useful than a C interface.


    If the library is C, I would use C. If it is C++, I would use C++,
    not C. I find it useful to use C++ library by C++ API as opposite
    to using a C++ lib by C API. BTW, plugins do know internals
    of the main program with libraries and can freely use them.

    One can just use C++, observing the same principles: ODR (put
    each plugin in its own namespace) and other code correctness.

    Technically build adjustments need to be made, see gcc faq on DSO.
    In short for RTTI type information is by default pulled in on ELF
    during linking, and loader does not see it, because linker does not
    export it. So to get exceptions, templates and other such beasts
    working, one needs to export all symbols (-E) and then the loader
    can do its job during dlopen().

    Edek
     
    Edek, Nov 3, 2011
    #13
  14. Waldek M.

    Edek Guest

    On 11/02/2011 11:03 PM, Paavo Helde wrote:
    > Edek<> wrote in
    > news:j8r0rb$mqa$:
    >> Why would you use nm? Why not make an extern C factory method
    >> returning an object with virtual methods you can call (same example)?

    >
    > The OP did not like extern "C" for some reason. I just clarified it is
    > technically possible to avoid it (though most probably not wise).


    I guess it depends whether we mean "any library", or a plugin.

    If the library is not a piece of unrelated software, one can write
    a plugin without a single extern "C" and without dlsym. They
    just need an API.

    A working example (linux, gcc):

    ----------------------- api.h ------------------
    #include <string>
    #include <map>

    using namespace std;

    struct Exc {};

    struct object
    {
    object () {};
    virtual void hello () const =0;
    virtual void throwExc () const = 0;
    };

    typedef map<string, object*> RegistryT;

    -------------------- lib.cpp ------------------
    #include "api.h"

    #include <iostream>

    using namespace std;

    extern RegistryT registry;

    namespace lib {

    struct MyObject : public object
    {
    void hello () const { cerr << "MyObject" << endl; }
    void throwExc () const { throw Exc() ; }
    };

    struct Registration
    {
    Registration () {
    registry["lib"] = new MyObject();
    }
    };

    static Registration reg;

    } // namespace
    --------------------- main.cpp ----------------
    #include "api.h"

    #include <dlfcn.h>
    #include <iostream>

    RegistryT registry;

    int main()
    {
    dlopen("./lib.so", RTLD_NOW | RTLD_GLOBAL);

    registry["lib"]->hello();

    try {
    registry["lib"]->throwExc();
    } catch (Exc& e) {
    cout << "caught Exc" << endl;
    }

    }

    Edek
     
    Edek, Nov 3, 2011
    #14
  15. Waldek M.

    Waldek M. Guest

    On Wed, 02 Nov 2011 17:03:04 -0500, Paavo Helde wrote:
    >> Why would you use nm? Why not make an extern C factory method
    >> returning an object with virtual methods you can call (same example)?

    >
    > The OP did not like extern "C" for some reason. I just clarified it is
    > technically possible to avoid it (though most probably not wise).


    Actually, that wasn't exactly what I wrote :) even though it's close.
    Yes, I do not like that in order to load C++ library from a C++
    executable, I need to provide a C interface on one side and use it on the
    other one.
    Yet, I'm just asking if there is a better way with the - I definitely
    wouldn't like to hack and depend on some particular version
    of particular compiler...

    Thanks anyway.

    Waldek
     
    Waldek M., Nov 3, 2011
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. flamesrock
    Replies:
    8
    Views:
    548
    Hendrik van Rooyen
    Nov 24, 2006
  2. Alex
    Replies:
    1
    Views:
    286
  3. Karsten Wutzke
    Replies:
    21
    Views:
    946
    Roedy Green
    Jun 29, 2007
  4. Ramon F Herrera
    Replies:
    21
    Views:
    702
    George Neuner
    Dec 4, 2007
  5. Pierre Yves
    Replies:
    2
    Views:
    521
    Pierre Yves
    Jan 10, 2008
Loading...

Share This Page