Anything new on run-time libraries loading?

[Waldek M.]

Hello,

The standard way for loading libraries on runtime (in this case, for
Linux/Unix systems) is dlsym/dlopen & friends.
The catch is, libraries written with C++ need to have some
kind of "C" interface (hence not mangled) that would allow access to C++
objects, at least supporting creation and destruction, like here:
http://tldp.org/HOWTO/C++-dlopen/thesolution.html

And my question: does the new C++ standard address this issue somehow?
Or, is there any more comfortable (still standard compliant) way to achieve
the same goal?

Best regards,
Waldek

 

[Goran]

Hello,

The standard way for loading libraries on runtime (in this case, for
Linux/Unix systems) is dlsym/dlopen & friends.
The catch is, libraries written with C++ need to have some
kind of "C" interface (hence not mangled) that would allow access to C++
objects, at least supporting creation and destruction, like here:http://tldp.org/HOWTO/C++-dlopen/thesolution.html

And my question: does the new C++ standard address this issue somehow?
Or, is there any more comfortable (still standard compliant) way to achieve
the same goal?

I think both C and C++ languages have no notion whatsoever of
libraries, *.so or *.lib, including latest C++ spec.

I personally think they should never get that. C-level
"interoperability" on library level works primarily because C is so
simple. He who needs something more than that, C, should reach for
some integration tech that defines needed binary-level compatibility.
I find it amazing that virtually no such tech, that is cross-platform
and in wide use, exist today. There's COM on Windows and that's about
it. Pffft...

Goran.

 

[Richard]

[Please do not mail me a copy of your followup]

Goran <[email protected]> spake the secret code

[...] There's COM on Windows and that's about it.

I believe the unix equivalent is CORBA.

 

[Ian Collins]

[Please do not mail me a copy of your followup]

Goran<[email protected]> spake the secret code

[...] There's COM on Windows and that's about it.

I believe the unix equivalent is CORBA.

s/unix/portable/

 

[Waldek M.]

I think both C and C++ languages have no notion whatsoever of
libraries, *.so or *.lib, including latest C++ spec.

It looks so, yes.

I personally think they should never get that. C-level
"interoperability" on library level works primarily because C is so
simple.

For instance, Python is a little bit more complex and yet it tries
to address inter-operability. Of course, C++ and P. are completely
different beasts and I'm far from trying any flaming whatsoever.

He who needs something more than that, C, should reach for
some integration tech that defines needed binary-level compatibility.
I find it amazing that virtually no such tech, that is cross-platform
and in wide use, exist today.

For C++, due to non-standarized name mangling, it's not a trivial thing.
That's why I hoped something has changed recently. Seems like it didn't.
Oh, well...

Best regards,
Waldek

 

[Edek]

I think both C and C++ languages have no notion whatsoever of
libraries, *.so or *.lib, including latest C++ spec.

Why would they? If they would, they would need to include llvm IR
binaries too (they are for example for JIT, still being c++).

I personally think they should never get that. C-level
"interoperability" on library level works primarily because C is so
simple. He who needs something more than that, C, should reach for
some integration tech that defines needed binary-level compatibility.
I find it amazing that virtually no such tech, that is cross-platform
and in wide use, exist today. There's COM on Windows and that's about
it. Pffft...

Speaking of dlsym/dlopen, there is binary compatibility. While I don't
know all platforms, compilers often agree to the same ABI, so it
is absolutely no problem to share data and call the code. All you
need is a little wrapper for _dynamically_ loaded libs, which are
plugins or whatever and this wrapper is not needed for every
interaction, just for one (or a couple) entry points used
as an object factory. So except for the factory stuff, not much
more is needed - an API of course, but it does not require
any C linkage, it can be just regular C++.

The whole point of that is not binary compatibility, but
the fact that in C++ one cannot call methods (constructors are
also callable) of unknown at compilation time name and arguments.

The possible problem with new does exist, but some things
may be decided by the policy of plugin integration, which
is needed anyway. There are many applications with plugins,
none of them uses CORBA or anything similar.

Edek

 

[Dombo]

Op 01-Nov-11 6:28, Richard schreef:

[Please do not mail me a copy of your followup]

Goran<[email protected]> spake the secret code

[...] There's COM on Windows and that's about it.

I believe the unix equivalent is CORBA.

More like the cross-platform equivalent; CORBA implantations are
available for other platforms as well (including Windows). There also
many other middleware solutions.

 

[Jorgen Grahn]

Op 01-Nov-11 6:28, Richard schreef:

[Please do not mail me a copy of your followup]

Goran<[email protected]> spake the secret code

[...] There's COM on Windows and that's about it.

I believe the unix equivalent is CORBA.

More like the cross-platform equivalent; CORBA implantations are
available for other platforms as well (including Windows). There also
many other middleware solutions.

Although at that point, you might find yourself saying "maybe a plain
C interface to the loadable part isn't such a bad idea after all" ...

/Jorgen

 

[Edek]

Adding a couple of extern "C" is as comfortable as you can get. Note that
the function marked extern "C" can be otherwise fully C++ in both the
definition and signature. There might be some complications, e.g.
throwing exceptions across the extern "C" border is not guaranteed to
work IIRC.

That is in the scope of factory methods, all other calls can be C++
(see the triangle example in the mentioned FAQ)

If you want to pass C++ objects, both libraries must be binary compatible
anyway, so in principle nothing prevents you sneaking out the mangled
name by the nm tool and using that in your dlsym() calls. However, this
is much more fragile than using extern "C" (there were times when gcc
changed its name mangling rules about every minor release, for example).

Why would you use nm? Why not make an extern C factory method returning
an object with virtual methods you can call (same example)?

Edek

 

[Edek]

Op 01-Nov-11 6:28, Richard schreef:

[Please do not mail me a copy of your followup]

Goran<[email protected]> spake the secret code
<46010469-4b5a-4c7c-bc88-f202f3df0841@q13g2000vbd.googlegroups.com> thusly:

[...] There's COM on Windows and that's about it.

I believe the unix equivalent is CORBA.

More like the cross-platform equivalent; CORBA implantations are
available for other platforms as well (including Windows). There also
many other middleware solutions.

Although at that point, you might find yourself saying "maybe a plain
C interface to the loadable part isn't such a bad idea after all" ...

Either I am missing something huge, or what are you talking about?

You can have a C++ interface (see the FAQ in OP, the triangle class
example)

Edek

 

[Dombo]

Op 02-Nov-11 10:03, Edek schreef:

Op 01-Nov-11 6:28, Richard schreef:
[Please do not mail me a copy of your followup]

Goran<[email protected]> spake the secret code
<46010469-4b5a-4c7c-bc88-f202f3df0841@q13g2000vbd.googlegroups.com>
thusly:

[...] There's COM on Windows and that's about it.

I believe the unix equivalent is CORBA.

More like the cross-platform equivalent; CORBA implantations are
available for other platforms as well (including Windows). There also
many other middleware solutions.

Although at that point, you might find yourself saying "maybe a plain
C interface to the loadable part isn't such a bad idea after all" ...

Either I am missing something huge, or what are you talking about?

You can have a C++ interface (see the FAQ in OP, the triangle class
example)

CORBA is rather complex, you have to be a bit of masochist to enjoy
dealing with this stuff.

 

[Jorgen Grahn]

Op 01-Nov-11 6:28, Richard schreef:
[Please do not mail me a copy of your followup]

Goran<[email protected]> spake the secret code
<46010469-4b5a-4c7c-bc88-f202f3df0841@q13g2000vbd.googlegroups.com> thusly:

[...] There's COM on Windows and that's about it.

I believe the unix equivalent is CORBA.

More like the cross-platform equivalent; CORBA implantations are
available for other platforms as well (including Windows). There also
many other middleware solutions.

Although at that point, you might find yourself saying "maybe a plain
C interface to the loadable part isn't such a bad idea after all" ...

Either I am missing something huge, or what are you talking about?

I'm saying that if your problem is the one in the original posting
(calling code in dlopen()ed libraries) you'd probably rather stick to
a C interface than go CORBA, or XMLsomething, or ...

(Perhaps the topic shifted without me noticing it, because I can't see
how CORBA solves that at all.)

You can have a C++ interface (see the FAQ in OP, the triangle class
example)

I missed that part -- but at a quick glance it doesn't seem much more
useful than a C interface.

/Jorgen

 

[Edek]

28, Richard schreef:
[Please do not mail me a copy of your followup]

Goran<[email protected]> spake the secret code
<46010469-4b5a-4c7c-bc88-f202f3df0841@q13g2000vbd.googlegroups.com> thusly:

[...] There's COM on Windows and that's about it.

I believe the unix equivalent is CORBA.

More like the cross-platform equivalent; CORBA implantations are
available for other platforms as well (including Windows). There also
many other middleware solutions.

Although at that point, you might find yourself saying "maybe a plain
C interface to the loadable part isn't such a bad idea after all" ...

Either I am missing something huge, or what are you talking about?

I'm saying that if your problem is the one in the original posting
(calling code in dlopen()ed libraries) you'd probably rather stick to
a C interface than go CORBA, or XMLsomething, or ...

I misaddressed my post, I think Goran started talking about
COM as if dlopen limited anybody to C:

He who needs something more than that, C, should reach for
some integration tech that defines needed binary-level compatibility.
I find it amazing that virtually no such tech, that is cross-platform
and in wide use, exist today. There's COM on Windows and that's about
it.

So maybe LoadLibrary on Windows limits usage to C and one needs COM,
I don't know if that is true or not. Yeah, platforms do differ.

(Perhaps the topic shifted without me noticing it, because I can't see
how CORBA solves that at all.)

CORBA has a lot of features like load balancing, entities and so on;
using CORBA for a plugin framework is 'a bit of' an overkill. I see
no point in using CORBA on a single machine at all, I see the
major downside of using anything that is so heavy for simple
method calls, mainly because of PITA for developers.

I missed that part -- but at a quick glance it doesn't seem much more
useful than a C interface.

If the library is C, I would use C. If it is C++, I would use C++,
not C. I find it useful to use C++ library by C++ API as opposite
to using a C++ lib by C API. BTW, plugins do know internals
of the main program with libraries and can freely use them.

One can just use C++, observing the same principles: ODR (put
each plugin in its own namespace) and other code correctness.

Technically build adjustments need to be made, see gcc faq on DSO.
In short for RTTI type information is by default pulled in on ELF
during linking, and loader does not see it, because linker does not
export it. So to get exceptions, templates and other such beasts
working, one needs to export all symbols (-E) and then the loader
can do its job during dlopen().

Edek

 

[Edek]

The OP did not like extern "C" for some reason. I just clarified it is
technically possible to avoid it (though most probably not wise).

I guess it depends whether we mean "any library", or a plugin.

If the library is not a piece of unrelated software, one can write
a plugin without a single extern "C" and without dlsym. They
just need an API.

A working example (linux, gcc):

----------------------- api.h ------------------
#include <string>
#include <map>

using namespace std;

struct Exc {};

struct object
{
object () {};
virtual void hello () const =0;
virtual void throwExc () const = 0;
};

typedef map<string, object*> RegistryT;

-------------------- lib.cpp ------------------
#include "api.h"

#include <iostream>

using namespace std;

extern RegistryT registry;

namespace lib {

struct MyObject : public object
{
void hello () const { cerr << "MyObject" << endl; }
void throwExc () const { throw Exc() ; }
};

struct Registration
{
Registration () {
registry["lib"] = new MyObject();
}
};

static Registration reg;

} // namespace
--------------------- main.cpp ----------------
#include "api.h"

#include <dlfcn.h>
#include <iostream>

RegistryT registry;

int main()
{
dlopen("./lib.so", RTLD_NOW | RTLD_GLOBAL);

registry["lib"]->hello();

try {
registry["lib"]->throwExc();
} catch (Exc& e) {
cout << "caught Exc" << endl;
}

}

Edek

 

[Waldek M.]

The OP did not like extern "C" for some reason. I just clarified it is
technically possible to avoid it (though most probably not wise).

Actually, that wasn't exactly what I wrote even though it's close.
Yes, I do not like that in order to load C++ library from a C++
executable, I need to provide a C interface on one side and use it on the
other one.
Yet, I'm just asking if there is a better way with the - I definitely
wouldn't like to hack and depend on some particular version
of particular compiler...

Thanks anyway.

Waldek