Virtual destructors and vtable layout

[Ole Nielsby]

Component models like COM, UNO, XPCOM etc. rely on
the vtable layout being the same across compilers.

I wonder what happens if virtual destructors are used.

I realise I shouldn't call virtual destructors across dll borders,
but it would be nice to be able to use them internally in
components, still.

My guess is, most compilers will deal with virtual destructors
like any other virtual functions, allocating vtable positions in
the order of declaration.

But then again, some compilers might use more than one
vtable slot, or allocate the virtual destructor at a fixed
negative offset for uniform treatment, or what do I know.

I know vtable-layout isn't officially standardised but without
de facto-standards for vtable layout, these component models
would be hard to deal with. So I wonder if virtual destructors
are dealt with in a de facto standard way, or will they break
the component models if used?

(Why I want to know: I'm trying to design a lightweight
component model for in-proc dll addins and I'd like to
have a virtual destructor in the object base class - but
not if it makes the vtable layout compiler dependent.)

 

[James Kanze]

Component models like COM, UNO, XPCOM etc. rely on the vtable
layout being the same across compilers.

It depends on which component model, surely. And whichever tool
is used to generate the component interface will conform to
whatever the rules are for that interface.

I wonder what happens if virtual destructors are used.

I realise I shouldn't call virtual destructors across dll
borders, but it would be nice to be able to use them
internally in components, still.

Why wouldn't you call them across .DLL borders. We have code in
places that would require it, and it's never caused any
problems. Across numerous versions of Solaris and Linux, and
some versions of Windows.

My guess is, most compilers will deal with virtual destructors
like any other virtual functions, allocating vtable positions
in the order of declaration.

Just about every compiler lays vtables out differently. You
can't expect to link code compiled with one compiler with code
compiled with another. (That's the reason why consciencious
compiler implementers ensure that their name mangling is
different from that of other compilers. So you get an error at
link time, rather than really strange behavior at runtime.)

But then again, some compilers might use more than one
vtable slot, or allocate the virtual destructor at a fixed
negative offset for uniform treatment, or what do I know.

You don't have to worry about it. It's the compiler's problem.

I know vtable-layout isn't officially standardised but without
de facto-standards for vtable layout, these component models
would be hard to deal with.

The vtable layout used by the C++ compiler has nothing to do
with what the component model does. None of the component
models I'm familiar with even use a vtable, except insofar as
they map to C++ code, in which case, they're just normal C++
code.

How could they, when you think about it? The usual vtable
implementation contains pointers into the local process; by
definition, a component model allows calling between processes,
and generally between machines with different architectures.
(Otherwise, why bother with it.)

So I wonder if virtual destructors are dealt with in a de
facto standard way, or will they break the component models if
used?

(Why I want to know: I'm trying to design a lightweight
component model for in-proc dll addins and I'd like to
have a virtual destructor in the object base class - but
not if it makes the vtable layout compiler dependent.)

If you're designing a plugin interface, it's up to you to decide
how to define it, but usually, at least under Windows and Unix,
it's defined in terms of C, to leverage off the system-wide C
language API.

 

[Ole Nielsby]

The vtable layout used by the C++ compiler has nothing to do
with what the component model does. None of the component
models I'm familiar with even use a vtable, except insofar as
they map to C++ code, in which case, they're just normal C++
code.

COM does rely on vtable layout. COM interfaces are declared
as pure virtual classes, all methods using stdcall convention, and
this works because most (if not all) C++ compilers for the MSW
use a very similar vtable layout.

How could they, when you think about it? The usual vtable
implementation contains pointers into the local process; by
definition, a component model allows calling between processes,
and generally between machines with different architectures.
(Otherwise, why bother with it.)

In COM, this is handled by proxies provided by system dlls,
and they rely on that same vtable layout.

If you're designing a plugin interface, it's up to you to decide
how to define it, but usually, at least under Windows and Unix,
it's defined in terms of C, to leverage off the system-wide C
language API.

Strictly speaking, COM and UNO interfaces are defined in
terms of an IDL but it's common practise to use C++ classes
(I don't think I've ever seen this stuff coded as explicit vtables
in C.)

 

[James Kanze]

COM does rely on vtable layout. COM interfaces are declared
as pure virtual classes, all methods using stdcall convention, and
this works because most (if not all) C++ compilers for the MSW
use a very similar vtable layout.

In other words, COM is pretty much unusable.

Seriously, you've got to be kidding. How does it work when the
components are on different machines? Running different
binaries with the functions at different addresses?

Either it's not really a component model, but just a means of
supporting linking different languages (which, admittedly, all
systems should provide), or you've misunderstood something. And
of course, when you link different languages, you tell the
compiler (that's what ``extern "xxx"'' is for, but I believe
VC++ uses some proprietary syntax instead), you (obviously) obey
the restrictions of the common interface (no pass by value for
Fortran, nothing but POD's for C, etc.), and the compiler does
whatever is necessary. Or you write a wrapper; Java can only
call code adhering to the JNI interface, so you if you want your
code to be callable from Java, you implement a wrapper for it
which adheres to the JNI interface.

 

[Ole Nielsby]

In other words, COM is pretty much unusable.

COM is a mess - but very usable.

Seriously, you've got to be kidding. How does it work when the
components are on different machines? Running different
binaries with the functions at different addresses?

By client proxies that communicate with the server components
using the network. The system libraries supply proxies for common
interfaces.

Either it's not really a component model, but just a means of
supporting linking different languages (which, admittedly, all
systems should provide), or you've misunderstood something.

I think COM does qualify as a component model, though
not a particularly consistent one. I know for sure that the
C++ headers used by C++ COM programs do rely on
the compiler using a specific layout for the vtables. This
may be undefined according to the C++ standard but it's
very widely used.

 

[James Kanze]

COM is a mess - but very usable.

Only in very limited contexts. And it probably should be
avoided even then, because its limitations prevent future
growth. If you can't easily move a component from one machine
to another, there's really no point in using a component
architecture, is there?

By client proxies that communicate with the server components
using the network. The system libraries supply proxies for
common interfaces.

In that case, who cares how the vtable is laid out. The proxy
is compiled in whichever language you want, and has the correct
vtable for that language and that compiler.

That's really how Corba works. (At that particular
level---there's a lot more to Corba. Which is nice if you
need it, and a major overhead if you don't.)

I think COM does qualify as a component model, though not a
particularly consistent one. I know for sure that the C++
headers used by C++ COM programs do rely on the compiler using
a specific layout for the vtables. This may be undefined
according to the C++ standard but it's very widely used.

But if the COM interface definition is in C++, you can compile
it to another language, using that language's binding. And if
you're using proxies, the vtable layout doesn't matter. And
can't; it makes no sense to say that the vtable layout is the
same on a Sparc as it is on a PC.

But if I understand you correctly, COM mixes two very different
concepts: a component model (using proxies, etc.), and a
specification for a mixed language platform specific API. The
latter needs a lot more than just a consistent vtable layout, of
course, but presumable, Microsoft ensures that all the rest
works as well in the tool chains it provides. (The problem is
that except for C++, and maybe C#, it doesn't provide any other
useful language. In mixed language programming, the "other"
language is usually either a legacy language---Cobol or
Fortran---or a language targetting a different model, like Perl,
Python, Ruby or Java.)

 

[Ole Nielsby]

[...]
But if I understand you correctly, COM mixes two very
different concepts: a component model (using proxies, etc.),
and a specification for a mixed language platform specific API.

I think that's correct.

The latter needs a lot more than just a consistent vtable layout,
of course, but presumable, Microsoft ensures that all the rest
works as well in the tool chains it provides.

It works with non-MS compilers, too. COM components can
be compiled by VC, Borland, GCC... and still work together
by virtue of compatible vtables.

XPCOM (the Mozilla/Firefox component model) and UNO
(the component model of OpenOffice) work much the same
way.

I think this style of interfacing was chosen because of its low
overhead when used in-process - all it takes to cross a
component border is a virtual call. So it fits with the
pay-per-use philosophy: you pay for the marshalling
only if you need it.

IMO it would be nice if C++ had some way of enforcing
a standard vtable layout.so that component models like
this could be used without relying on what is strictly
speaking undefined beaviour.

 

[Bo Persson]

[...]
But if I understand you correctly, COM mixes two very
different concepts: a component model (using proxies, etc.),
and a specification for a mixed language platform specific API.

I think that's correct.

The latter needs a lot more than just a consistent vtable layout,
of course, but presumable, Microsoft ensures that all the rest
works as well in the tool chains it provides.

It works with non-MS compilers, too. COM components can
be compiled by VC, Borland, GCC... and still work together
by virtue of compatible vtables.

XPCOM (the Mozilla/Firefox component model) and UNO
(the component model of OpenOffice) work much the same
way.

I think this style of interfacing was chosen because of its low
overhead when used in-process - all it takes to cross a
component border is a virtual call. So it fits with the
pay-per-use philosophy: you pay for the marshalling
only if you need it.

IMO it would be nice if C++ had some way of enforcing
a standard vtable layout.so that component models like
this could be used without relying on what is strictly
speaking undefined beaviour.

If becomes implementation specific behaviour, when the compilers
support it. As far as I know, this is only done for x86 hardware.

The problem with defining implementation details is that is goes WAY
down to the underlying hardware. One example of what not to do is Java
specifying IEEE floating point, even for hardware that doesn't have
it. This has forced IBM to add another set of FP hardware to its
mainframes. Very expensive!

Would you like them to support Windows COM compatible vtables on z/OS
as well?


Bo Persson

 

[Erik Wikström]

[...]
But if I understand you correctly, COM mixes two very
different concepts: a component model (using proxies, etc.),
and a specification for a mixed language platform specific API.

I think that's correct.

The latter needs a lot more than just a consistent vtable layout,
of course, but presumable, Microsoft ensures that all the rest
works as well in the tool chains it provides.

It works with non-MS compilers, too. COM components can
be compiled by VC, Borland, GCC... and still work together
by virtue of compatible vtables.

IMO it would be nice if C++ had some way of enforcing
a standard vtable layout.so that component models like
this could be used without relying on what is strictly
speaking undefined beaviour.

Strictly speaking it is not undefined behaviour, it is just not defined
in the C++ standard. Which is OK, most development probably relies on
stuff not specified in C or C++ (POSIX comes to mind).

Actually I prefer it that way, the language standard should try not to
specify implementation-details unless necessary, and leave it up to each
platform to provide such details and more (such as how to interop with
other languages/components/applications/etc.).

 

[James Kanze]

[...]
The latter needs a lot more than just a consistent vtable
layout, of course, but presumable, Microsoft ensures that
all the rest works as well in the tool chains it provides.

It works with non-MS compilers, too. COM components can be
compiled by VC, Borland, GCC... and still work together by
virtue of compatible vtables.

XPCOM (the Mozilla/Firefox component model) and UNO (the
component model of OpenOffice) work much the same way.

Do they? Firefox requires that its plugins be compiled with a
specific compiler?

I think this style of interfacing was chosen because of its
low overhead when used in-process - all it takes to cross a
component border is a virtual call.

But that's the case with the Corba implementations I've used as
well. When the language binding supports it.

So it fits with the pay-per-use philosophy: you pay for the
marshalling only if you need it.

IMO it would be nice if C++ had some way of enforcing a
standard vtable layout.so that component models like this
could be used without relying on what is strictly speaking
undefined beaviour.

IIUC, what you're requesting is that the C++ standard impose
implementation details on other languages. I don't think it's
possible.

 

[Yakov Gerlovin]

But that's the case with the Corba implementations I've used as
well. When the language binding supports it.

May I ask which ORB you're referring to an how it is implemented?
Specifically, I'd like to know:
1. Do you need a different declaration in IDL, something like "local"
2. May you use the same interface both as local (without marshalling)
and remote?
3. Do you get 2 different sets of skeleton and proxies for local and
remote?

Thanks in advance,
Yakov

 

[James Kanze]

May I ask which ORB you're referring to an how it is implemented?

I've used several. I've not really looked into the
implementation, but I do know that it doesn't marshal if the
object is located in the same process.

Specifically, I'd like to know:
1. Do you need a different declaration in IDL, something like "local"
No.

2. May you use the same interface both as local (without marshalling)
and remote?

Yes. At least I think so (it's been some time ago); the
decision is runtime; depending on where the object is located,
you'll end up with the marshalling proxy, or the actual object.

3. Do you get 2 different sets of skeleton and proxies for local and
remote?

No. IIRC, you don't even have a proxy for local objects.

 

[Ole Nielsby]

[...]
The latter needs a lot more than just a consistent vtable
layout, of course, but presumable, Microsoft ensures that
all the rest works as well in the tool chains it provides.

It works with non-MS compilers, too. COM components can be
compiled by VC, Borland, GCC... and still work together by
virtue of compatible vtables.

XPCOM (the Mozilla/Firefox component model) and UNO (the
component model of OpenOffice) work much the same way.

Do they? Firefox requires that its plugins be compiled with a
specific compiler?

I'm not deep into XPCOM but UNO/OpenOffice is compiler
dependent, and the MSW build is tied to VC6 which is sort
of tragic for an application that's supposed to challenge the
Microsoft monopoly in office software, and a major deterrent
from using UNO IMO.

This dependency isn't caused by vtable issues. It has to do with
components' use of rtl, and an issue with COM interop that
depends on ATL.

 

[Stephen Horne]

On Oct 10, 5:21 pm, "Ole Nielsby"


Why wouldn't you call them across .DLL borders. We have code in
places that would require it, and it's never caused any
problems. Across numerous versions of Solaris and Linux, and
some versions of Windows.

There are issues with some compilers, but probably only old ones. I
remember problems with Borland C++ 5.02. Each DLL ended up with a
different heap. It's possible that linker options or similar were to
blame, but the easiest thing at the time was to ensure that data
always got deleted by the same DLL that newed it - usually not a big
deal.

Not something I was particularly involved in - just a rule I had to
work with at the time.

 

[Stephen Horne]

In other words, COM is pretty much unusable.

Some compilers have been made COM-compliant, but COM was never
designed to be portable - it was meant to be a Microsoft specific
system. .NET is better, but it's still something that compilers comply
with rather than visa versa.

Anyway, COM is a fair bit older than the C++ standard and from a time
when Microsoft didn't care much about any standards but their own.

Originally, I doubt anyone even considered the possibility of COM
supporting compilers other than those by Microsoft. OTOH, one of the
early design constraints was that it should be possible (but not easy)
to write a component in C. That is, to manually set up your own vtable
etc. At the time, it was felt that there was no guarantee that C++
would really catch on.

Seriously, you've got to be kidding. How does it work when the
components are on different machines? Running different
binaries with the functions at different addresses?

COM+ and later have support for this kind of thing. The support makes
assumptions about the compiler, but basically it translates the data
to an intermediate form anyway. Part of COM is a data definition DSL
that tells the framework where the fields are and how they're
represented. The .NET framework has some rules which compilers must
follow, and support for data definition is part of the reason for
language features such as attributes in C# and Managed C++.

Very basic COM doesn't need data definitions, but that's because very
basic COM only supported components that were simple DLLs linked
within the same process. You could basically acquire interface
pointers (using more or less the Java sense of the word interface) and
test whether those interfaces could be safely cast to other forms, and
there were some reference counting hassles, but it's a pretty thin API
really. If you can remember the words IUnknown and QueryInterface,
you're half-way there.

I assume it was originally about interop with Basic, but don't quote
me. OLE2 for office was probably the start of COMs evolution into a
do-everything object model, followed by ActiveX for internet explorer
and so on. COM+ didn't happen until Windows 2000 I think - parts of
the framework are part of the OS.

Don't recall whether it was originally possible to write a COM
component in Basic, or only to use one, but automation of the calling
side was meant to be one of the big advantages of VB. COM-based
database access and interfacing with Crystal Reports were early killer
apps.

 

[James Kanze]

There are issues with some compilers, but probably only old
ones. I remember problems with Borland C++ 5.02. Each DLL
ended up with a different heap. It's possible that linker
options or similar were to blame, but the easiest thing at the
time was to ensure that data always got deleted by the same
DLL that newed it - usually not a big deal.

Not something I was particularly involved in - just a rule I
had to work with at the time.

That has nothing to do with COM, but is related to the way
dynamic linking works under Windows and some poorly choosen
defaults in the linker. From what I understand, the problem is
that the library which contains malloc/free is linked
statically by default, in each dynamic object. And that it is
sufficient to use the dynamically linked version of this library
to avoid the problem.