dynamic library loading, missing symbols

[dfj225]

While my question doesn't pertain specifically to python programming,
it is a result of developing a python module, so I'm hoping someone
here might have experience with this issue.

So, first a little background to how the system works right now. I am
developing a module for Python. The original code is written in C++ and
I am creating a wrapper using Boost.Python. The C++ code for the module
makes calls into a library that in turn dynamically loads a library
(using dlopen). This dynamically loaded library, in turn, expects to
find a symbol in the code that loaded it (using extern it declares a
variable). However, the linker fails at runtime here because it can't
find this symbol. The variable in question does exist in the C++ code
that in does dlopen.

The reason for the complication is that I don't have control over how
the library does dlopen() or how the code that calls dlopen was
compiled. I am, however, able to control the build process for the
Boost.Python wrapper and the original C++ code that the Boost.Python
wraps. I've tried as many linker tricks as I can think of to get this
to work. Both the boost wrapper and the C++ code that it wraps are
built using --export-dynamic.

Is there a way to set at runtime what directories or libraries the
linker should search for these symbols? I have set LD_LIBRARY_PATH
correctly, but that didn't seem to affect anything.

For reference, I am running on Gentoo linux 2.6.11.12 with gcc 3.4.4

I'm interested in any ideas that might help, but the ideal one should
work on any *nix system and not just linux.

(Please don't suggest that I change the operation of the code that is
doing dlopen, as it is 3rd party and not something that would be in our
best interest to work on).

If you think I need to provide more information, I'd be happy to try
and clarify things.

Thanks,
~Doug

 

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

However, the linker fails at runtime here because it can't
find this symbol. The variable in question does exist in the C++ code
that in does dlopen.

Did you verify, using nm -D, that the symbol is indeed present in
the shared object, not just in the source code?

The reason for the complication is that I don't have control over how
the library does dlopen() or how the code that calls dlopen was
compiled.

What flags are given to that dlopen call?

Is there a way to set at runtime what directories or libraries the
linker should search for these symbols?

No. The dynamic linker doesn't search files to resolve symbols; instead,
it searches the process' memory. It first loads the referenced shared
libraries (processing the DT_NEEDED records in each one); that uses
the LD_LIBRARY_PATH. Then, symbol resolution needs to find everything
in the libraries that have already been loaded.

There are various ways to control which subset of in-memory symbols
the dynamic linker considers: the RTLD_GLOBAL/RTLD_LOCAL flags play
a role, the -Bsymbolic flag given to the static linker has an impact,
and so does the symbol visibility (hidden/internal/protected).

Regards,
Martin