Why export templates would be useful

[Brian]

  Some people put the template implementations in a separate file with a
distinctive extension such as .tcc, and then use #include
"implementation.tcc" at the end of the header file. This allows keeping
the source code in two files, but there's no separate compilation unit
for the template.

I have it in a .hh and a .cc, and the .cc is included.
I guess I could change the extension to .tcc.

  Another option is that if you know in advance all the types that your
template will be used with, you can use explicit instantiation in a true
source file to emulate export templates. This is a rather limited
technique, obviously.

That might work for me. I have three possible types that
the template could be used with and if an application
needs to communicate with big and little endian machines,
it uses two of those types. Thanks for the suggestion.


Brian Wood
http://webEbenezer.net
(651) 251-9384

 

[Ian Collins]

"In the past we added an option that allows users to
specify whether they want the code in header-only
(integrated) form or as two separate files. This was
in part because of some comments made by Kanze about
build times I think. Any thoughts on how to reconcile
this template use with that support? I'm not sure if
it's possible to keep supporting the separate header
and implementation approach with these changes."

Some compilers have rules for locating template implementations, usually
the compiler will look for a .cc or .C file in the same directory as the
matching declaration file.

This unless your build system is smart, doesn't change build times or
reduce coupling, but it does provide some degree of separation. The
dilemma with simple build systems is do you rebuild any file the
includes the header if the implementation changes, or not. If you
do,you may as well inline the implementation, if you don't you miss
changes unless you "touch" the header.

I'm sure these difficulties can be overcome which would go some way to
bridging the gap between exported templates and inline code.

 

[Brian]

That's a catchy sound bite, and I rarely disagree with James. But
indulge me a moment to say why I think it isn't accurate or fair.

First, "getting rid of export" didn't take any person-hours away from
C++0x. The paper Tom and I wrote was debated at exactly one meeting
(Oxford, April 2003) before C++0x work had begun. Since 2003 and
during the entire development of C++0x, as far as I'm aware, there has
been exactly zero effort to get rid of export other than the
submission of a simple 'please deprecate export' national body comment
on the CD ballot a year or so ago. (Incidentally, I wasn't even aware
of that comment for a while since I hadn't read through the CD
comments closely -- this occurred during the year that Bill Plauger
was convener and running things -- and so I was surprised to hear
after the fact that at Frankfurt meeting last summer the Core subgroup
had briefly discussed whether to do something about export.)

If you s/getting rid of export/getting in concepts/ I *might* agree
just slightly, because concepts was in retrospect the biggest single
risk factor and time sink in the whole C++0x process that drained
resources from many other proposals. But even blaming concepts
wouldn't really be fair. For one thing, you can only blame feature X
for draining resources from feature Y if the same people are working
on it, which wasn't the case here (and modules never got to a point
where it needed broad committee time). For another, some of us hoped
that concepts would actually give us many of the benefits of export,
without the unworkability of export itself.

As with most proposals, the biggest thing holding back the modules
proposal was itself -- that it wasn't ready for standardization,
because it was a very interesting and inventive proposal but with no
field experience. Daveed Vandevoorde in particular deserves credit for
doing a lot of interesting work and progressing the proposal, and I
hope he resumes it post-C++0x. But a paper design with a prototype
only goes so far, and the lack of field experience made it an easy
early scope/feature cut back in October 2006 when we saw the mountain
of other C++0x work still to be done (notably the large amount of
effort that the concepts proposal would still need, which in
retrospect we actually greatly underestimated).

If only concepts had been cut in 2006 and export either left
alone or replaced by modules in C++0x. The way it has turned
out is disappointing.


Brian Wood
http://webEbenezer.net
(651) 251-9384

 

[James Kanze]

That's a catchy sound bite, and I rarely disagree with James.
But indulge me a moment to say why I think it isn't accurate
or fair.

Of course. It was meant as a catchy sound bite, and not a
technical discussion. It was in response to something which was
basically similar in tone. In practice, people (and companies)
expend effort on what they decide, and not expending it on one
side doesn't mean expending it on another.

The time for technical discussions on this subject is over. I
disagree with the decision, but the compiler vendors have voted,
and unless some compiler implementors really do implement
export, it really doesn't matter what the standard decides to
do. I feel somewhat "had" by the compiler vendors (all of
them), because export addresses a real problem, and I'd have
rather the committee left it alone until a better solution to
the problem was adopted. I also feal that the fact that
implementers have refused to implement it (except EDG) sets a
bad precedent: what new feature in the next standard will they
decide not to implement? What's the point in even having a
standard if implementers only choose from it, and don't attempt
to implement everything?

[...]

As with most proposals, the biggest thing holding back the modules
proposal was itself -- that it wasn't ready for standardization,

I understand this. For the present, it isn't really ripe for
standardization, because it is so new. That argument, of
course, could be offered for most of the "improvements" in the
1998 standard: namespaces, two phase look-up, etc. And
"export". There are serious arguments why all of these should
NOT have been in the 1998 standard. But that's water under the
bridge, and presumably, the committee has learned from the
problems it caused first time around: export, obviously, but in
many ways, two phased lookup has caused more problems.

In many ways, two phased lookup is like export: it attempts to
solve a real problem; it may not be the best, or even a good
solution; and a lot of people don't consider it an important
problem. The main difference is that compiler writers did try
to implement it, with different degrees of success, with the
results that even today, it causes no end of portability
problems. (At least with export, you can say that all of the
vendors are at the same level.)

[...]

I think it's fair to say that nearly everyone (including me)
still wants modules to succeed to get right rid of the header
file text inclusion model, but that most people (including me)
are deeply afraid of messing with something as fundamental as
the C++ build model without real world experience.

I totally agree with that point of view. I just think that
export could have been left in until we got there. Practically,
it makes no real difference (since I can't use it in portable
code), but it is a way of formally acknowledging the problem,
and perhaps motivating some to work harder on modules (so that
they could get rid of export).

But there are certainly more important issues, and I wouldn't
vote against the FCD on those grounds.

 

[James Kanze]

Some compilers have rules for locating template
implementations, usually the compiler will look for a .cc or
.C file in the same directory as the matching declaration
file.

This unless your build system is smart, doesn't change build
times or reduce coupling, but it does provide some degree of
separation.

It doesn't reduce coupling, but such systems are often
associated with a repository, which can reduce build times
significantly. (And also increase the risk of missing a
necessary rebuild---CFront suffered enormously from this
problem.)

The dilemma with simple build systems is do you rebuild any
file the includes the header if the implementation changes, or
not. If you do, you may as well inline the implementation, if
you don't you miss changes unless you "touch" the header.

FWIW: from what I understand (only from hearsay, but some of the
people describing this to me actually worked on the compiler in
question), Visual Age C++ didn't have this problem. It kept a
repository of compiled source code, at the function level, and
only fully recompiled if the actual token sequence of the
function/class/etc. was modified. (IIUC, the editor was
integrated into the system, and when you saved the edited file,
it tokenized, compared the results with a saved version, and
invalidated the repository entries if something changed.)

 

[Balog Pal]

In many ways, two phased lookup is like export: it attempts to
solve a real problem; it may not be the best, or even a good
solution; and a lot of people don't consider it an important
problem. The main difference is that compiler writers did try
to implement it, with different degrees of success, with the
results that even today, it causes no end of portability
problems. (At least with export, you can say that all of the
vendors are at the same level.)

I always thought the 2-phase lookup was introduced *for* the sake of export.
Without it there is little sense.

Then with export gone that feature still remains and will keep haunting.
Because, as you say compiler writers did implement it to some degree. Too
bad.

Sometime I keep wondering about a different universe, where C++ was
standardized in a different way. Mostly sticking to ARM stuff in the first
run. In my vision the standard was issued little earlier (end '97 latest),
and was covered in compilers full, in short time. All DRs got resolved in
the next 2-3 years, while the evolution group was working on new features,
and libraries like boost were born in the world -- using more resources on
actual stuff instead of fighting the diverse compilers.

By 2003 the next release of the standard was not just a patch but introduced
a more advanced library, and possibly even core changes we're looking ahead
(threads, ....).

It's unpleasant to wake up.

I totally agree with that point of view. I just think that
export could have been left in until we got there. Practically,
it makes no real difference

I see realistic a difference. Having a compiler that claims 100% conforming
has a drive on the makers and on the market. While if there is an
"except" list, it is less interesting how many elements are there.

Having export as a fixed item on that list IMO prevents keeping it clear of
other elements.

 

[James Kanze]

"James Kanze" <[email protected]>

[...]

I see realistic a difference. Having a compiler that claims
100% conforming has a drive on the makers and on the market.

Apparently not. If that were the case, we'd have export.

 

[Jerry Coffin]

[ ... ]

Since when is improved encapsulation a "minimal" benefit.

First of all, I qualified that, saying "fairly minimal", not just
"minimal". Second, I started out by saying that they sometimes
provide some benefit.

So, the question basically comes down to how often the encapsulation
that's available without export is inadequate, but with export it is
adequate. At least in my experience, the answer to that is, "not very
often".

And
from a user point of view: I get the benefit, but the cost is
paid by the compiler vendor. (I know that that's somewhat
simplified, but you get the idea.)

I'd say it's simplified to the point of being dead wrong. The notion
that the vendor pays instead of the consumer paying is just plain
nonsensical. The best you hope for is that the cost can be amortized
over a lot of consumers so individually you can get away with paying
only a fraction of the cost.

If it was easy to go out and hire the talent to implement "export",
that would be a reasonable viewpoint -- but we both know that's not
the case at all. In reality, the people to work on the compiler will
usually be the limiting factor -- which means compiler features tend
to be pretty much a zero-sum game. Consumers can reasonably choose
_which_ features get the most effort, but effort spent on one feature
almost inevitably takes away from the effort that can be put into
other features.

Therefore, as a consumer my interest is in finding the features that
provide me with the highest benefit to effort ratio. If I were to
write a list based on that, I don't think "export" would make it into
the first 10 items -- and quite possibly not even the first 10 pages.

[ ... ]

Something better has already been proposed, but not adopted, for
lack of time. Perhaps if as much effort had been put into it as
was put into getting rid of export. (Had the proposal been to
replace export with something better, I'd not be opposed.)

I, for one, would certainly _prefer_ a "make before break" change --
but I think it's unrealistic. I'd say a large part of the reason time
wasn't found for Daveed's module proposal was precisely because
export was present -- and I think almost any other such proposal
would have been treated the same was as long as export remained in
the standard.

 

[Balog Pal]

"James Kanze" <[email protected]>
[...]

I totally agree with that point of view. I just think that
export could have been left in until we got there. Practically,
it makes no real difference

I see realistic a difference. Having a compiler that claims
100% conforming has a drive on the makers and on the market.

Apparently not. If that were the case, we'd have export.

Drive is just one force. It was not enough to balance export. It may be
enough for all the rest. We will see.

 

[James Kanze]

[ ... ]

Since when is improved encapsulation a "minimal" benefit.

First of all, I qualified that, saying "fairly minimal", not
just "minimal". Second, I started out by saying that they
sometimes provide some benefit.

So, the question basically comes down to how often the
encapsulation that's available without export is inadequate,
but with export it is adequate. At least in my experience, the
answer to that is, "not very often".

Only in cases where you might want to use templates for
something other than very stable basics.

At present, we have a chicken and egg problem. Without export,
you can't really use templates for anything but the most stable
basics. So you don't try, and you don't know whether it would
be really useful in your application or not.

I'd say it's simplified to the point of being dead wrong. The
notion that the vendor pays instead of the consumer paying is
just plain nonsensical. The best you hope for is that the cost
can be amortized over a lot of consumers so individually you
can get away with paying only a fraction of the cost.

You mean like in the case of g++.

Seriously, development cost is a negligible part of most modern
compilers (g++ excepted). Far outweighed by commercialization
and distribution. And support---adding export would probably
reduce support costs slightly. (Making templates behave by
default as if they had been declared export would reduce them
significantly, but that wouldn't be conform.)

If it was easy to go out and hire the talent to implement
"export", that would be a reasonable viewpoint -- but we both
know that's not the case at all.

I know that EDG actually offered to help other implementors, and
provide the knowledge they had acquired doing the first
implementation. I also know that companies like Microsoft and
Sun already have some very qualified people working for them;
the fact that they choose to use these people for things like
CLI or Java is a separate problem.

In reality, the people to work on the compiler will usually be
the limiting factor -- which means compiler features tend to
be pretty much a zero-sum game. Consumers can reasonably
choose _which_ features get the most effort, but effort spent
on one feature almost inevitably takes away from the effort
that can be put into other features.

Yes and no. What other features have been added to, say Visual
Studios, since version 7. Some important bug fixes, perhaps,
but the basic compiler hasn't changed that much.

Therefore, as a consumer my interest is in finding the
features that provide me with the highest benefit to effort
ratio. If I were to write a list based on that, I don't think
"export" would make it into the first 10 items -- and quite
possibly not even the first 10 pages.

Funny. I would have put it very close to the top of the
unimplemented features. It's lack severely handicaps my use of
templates.

[ ... ]

Something better has already been proposed, but not adopted,
for lack of time. Perhaps if as much effort had been put
into it as was put into getting rid of export. (Had the
proposal been to replace export with something better, I'd
not be opposed.)

I, for one, would certainly _prefer_ a "make before break"
change -- but I think it's unrealistic. I'd say a large part
of the reason time wasn't found for Daveed's module proposal
was precisely because export was present -- and I think almost
any other such proposal would have been treated the same was
as long as export remained in the standard.

As far as I can tell, most, if not all vendors had dropped
export long before Daveed's proposal. (It's curious to note
that Daveed is also the only person to have actually implemented
export. I have no idea if that signifies anything, but it's
vaguely possible that his experience with implementing export
gave him a better perception of what could or could not be
done.)

 

[Jerry Coffin]

[ ... ]

Only in cases where you might want to use templates for
something other than very stable basics.

At present, we have a chicken and egg problem. Without export,
you can't really use templates for anything but the most stable
basics. So you don't try, and you don't know whether it would
be really useful in your application or not.

Yes, but I'm basing my comments on a project where we _did_ have
export -- we started with Comeau as the compiler for the project
(assuming that other conforming implementations would be
forthcoming). There's never been any (at least intentional)
concession to less-conforming compilers made in that project.

I suppose it _could_ be from lack of imagination in using them, but
neither management nor coding in that project seems to have been
affected to any major degree by the presence of export. Of course
there is another limiting factor: one of the original targets for
export was libraries -- and of course, none of the libraries out
there uses export (at all). Maybe if they did, I'd see more real
result from export -- but of course, they not only don't exist, but
at this point it's pretty unlikely that they ever will.

[ ... ]

I know that EDG actually offered to help other implementors, and
provide the knowledge they had acquired doing the first
implementation. I also know that companies like Microsoft and
Sun already have some very qualified people working for them;
the fact that they choose to use these people for things like
CLI or Java is a separate problem.

I'm not sure it's a separate problem at all -- they're looking at
basically the same situation I was talking about, but at a somewhat
higher level -- instead of just features in a compiler, at which
products give the biggest payoff for a given investment. The big
difference, however, is that they're looking at the payoff to them,
not to me.

[ ... ]

Yes and no. What other features have been added to, say Visual
Studios, since version 7. Some important bug fixes, perhaps,
but the basic compiler hasn't changed that much.

I'm going to assume you mean since 7.1 -- that was the major step
forward (for MS) in terms of conformance. Actually, there's been
quite a bit since then, though it's mostly a large number of fixes,
each of which is fairly minor in itself. Collectively I'd say they
represent quite a bit of work *and* a substantial improvement in the
compiler (and library).

Funny. I would have put it very close to the top of the
unimplemented features. It's lack severely handicaps my use of
templates.

If you look at it purely in terms of "unimplemented features", I'd be
forced to agree -- but I think that's too narrow a view. The only
other major feature that's unimplemented is exception specifications,
and I'm just as happy with their absence. As such, export wins by
default.

From my viewpoint, however, some improvements in 64-bit code
generations (just for one example) would qualify as more important
than probably any actual "feature" would. That's not to say that the
64-bit code generation is particularly disappointing or that
improvements are critical, just that export is still less important.

[ ... ]

As far as I can tell, most, if not all vendors had dropped
export long before Daveed's proposal. (It's curious to note
that Daveed is also the only person to have actually implemented
export. I have no idea if that signifies anything, but it's
vaguely possible that his experience with implementing export
gave him a better perception of what could or could not be
done.)

Maybe -- but my impression has been that most compiler implementors
were pretty well aware from the beginning of the difficulty of
implementing export. Modules have been implemented often enough in
enough other languages that I don't think one needs experience with
export to put them in the "much more implementable" class. From there
it's mostly a question of fitting what's been done already with the
way C++ works otherwise to get at least a much more reasonable
possibility.

 

[Brian]

[ ... ]

Only in cases where you might want to use templates for
something other than very stable basics.

At present, we have a chicken and egg problem.  Without export,
you can't really use templates for anything but the most stable
basics.  So you don't try, and you don't know whether it would
be really useful in your application or not.

Yes, but I'm basing my comments on a project where we _did_ have
export -- we started with Comeau as the compiler for the project
(assuming that other conforming implementations would be
forthcoming). There's never been any (at least intentional)
concession to less-conforming compilers made in that project.

I suppose it _could_ be from lack of imagination in using them, but
neither management nor coding in that project seems to have been
affected to any major degree by the presence of export. Of course
there is another limiting factor: one of the original targets for
export was libraries -- and of course, none of the libraries out
there uses export (at all). Maybe if they did, I'd see more real
result from export -- but of course, they not only don't exist, but
at this point it's pretty unlikely that they ever will.

[ ... ]

I know that EDG actually offered to help other implementors, and
provide the knowledge they had acquired doing the first
implementation.  I also know that companies like Microsoft and
Sun already have some very qualified people working for them;
the fact that they choose to use these people for things like
CLI or Java is a separate problem.

I'm not sure it's a separate problem at all -- they're looking at
basically the same situation I was talking about, but at a somewhat
higher level -- instead of just features in a compiler, at which
products give the biggest payoff for a given investment. The big
difference, however, is that they're looking at the payoff to them,
not to me.

Once they have more than one language things get complicated.
If they put a lot of work into their C++ implementation those
who are using their other languages may wonder what the point
is of paying extra for less features. They also can't really
implement anything too close to export in their other languages
as then the pressure would just increase for them to add it to
their C++ implementation. I think they try to walk a tight-
rope of sorts and that's one reason why we find the results to
be barely satisfactory. If you're happy with their C++
implementation, what hope do they have of moving you into
their other stuff? I don't understand it all, but have the
feeling they aren't very open about their motives.


Brian Wood
http://webEbenezer.net
(651) 251-9384

[ ... ]

Yes and no.  What other features have been added to, say Visual
Studios, since version 7.  Some important bug fixes, perhaps,
but the basic compiler hasn't changed that much.

I'm going to assume you mean since 7.1 -- that was the major step
forward (for MS) in terms of conformance. Actually, there's been
quite a bit since then, though it's mostly a large number of fixes,
each of which is fairly minor in itself. Collectively I'd say they
represent quite a bit of work *and* a substantial improvement in the
compiler (and library).

Funny.  I would have put it very close to the top of the
unimplemented features.  It's lack severely handicaps my use of
templates.

If you look at it purely in terms of "unimplemented features", I'd be
forced to agree -- but I think that's too narrow a view. The only
other major feature that's unimplemented is exception specifications,
and I'm just as happy with their absence. As such, export wins by
default.

From my viewpoint, however, some improvements in 64-bit code
generations (just for one example) would qualify as more important
than probably any actual "feature" would. That's not to say that the
64-bit code generation is particularly disappointing or that
improvements are critical, just that export is still less important.

[ ... ]

As far as I can tell, most, if not all vendors had dropped
export long before Daveed's proposal.  (It's curious to note
that Daveed is also the only person to have actually implemented
export.  I have no idea if that signifies anything, but it's
vaguely possible that his experience with implementing export
gave him a better perception of what could or could not be
done.)

Maybe -- but my impression has been that most compiler implementors
were pretty well aware from the beginning of the difficulty of
implementing export. Modules have been implemented often enough in
enough other languages that I don't think one needs experience with
export to put them in the "much more implementable" class. From there
it's mostly a question of fitting what's been done already with the
way C++ works otherwise to get at least a much more reasonable
possibility.

 

[David Vandevoorde]

I mostly agree with Herb, but a couple of remarks nonetheless...

That's a catchy sound bite, and I rarely disagree with James. But
indulge me a moment to say why I think it isn't accurate or fair.

First, "getting rid of export" didn't take any person-hours away from
C++0x. The paper Tom and I wrote was debated at exactly one meeting
(Oxford, April 2003) before C++0x work had begun. Since 2003 and
during the entire development of C++0x, as far as I'm aware, there has
been exactly zero effort to get rid of export other than the
submission of a simple 'please deprecate export' national body comment
on the CD ballot a year or so ago. (Incidentally, I wasn't even aware
of that comment for a while since I hadn't read through the CD
comments closely -- this occurred during the year that Bill Plauger
was convener and running things -- and so I was surprised to hear
after the fact that at Frankfurt meeting last summer the Core subgroup
had briefly discussed whether to do something about export.)

If you s/getting rid of export/getting in concepts/ I *might* agree
just slightly, because concepts was in retrospect the biggest single
risk factor and time sink in the whole C++0x process that drained
resources from many other proposals. But even blaming concepts
wouldn't really be fair.

Again: I strongly agree with this. Some of us weren't _that_ enthused
about getting concepts into C++0x (see my voting record ;-P), but the
work was admirable, worthwhile, and remains promising. I'm still
hoping that a future standard will see them (provided the programming
model is attractive).

For one thing, you can only blame feature X
for draining resources from feature Y if the same people are working
on it, which wasn't the case here (and modules never got to a point
where it needed broad committee time). For another, some of us hoped
that concepts would actually give us many of the benefits of export,
without the unworkability of export itself.

Really? You expected concepts to provide separate compilation of
templates?

I don't think that's a reasonable expectation. At best, concepts
could "optimize" exported templates because instantiation might be
guaranteed to succeed (though even that was never true for all
templates).

As with most proposals, the biggest thing holding back the modules
proposal was itself -- that it wasn't ready for standardization,
because it was a very interesting and inventive proposal but with no
field experience. DaveedVandevoordein particular deserves credit for
doing a lot of interesting work and progressing the proposal, and I
hope he resumes it post-C++0x. But a paper design with a prototype
only goes so far, and the lack of field experience made it an easy
early scope/feature cut back in October 2006 when we saw the mountain
of other C++0x work still to be done (notably the large amount of
effort that the concepts proposal would still need, which in
retrospect we actually greatly underestimated).
Agreed.

I think it's fair to say that nearly everyone (including me) still
wants modules to succeed to get right rid of the header file text
inclusion model, but that most people (including me) are deeply afraid
of messing with something as fundamental as the C++ build model
without real world experience.

In the end I was the sole voice that spoke up against the removal of
export: I tried to argue that they should remain in the WP until a
replacement mechanism is added (and I think modules are the most
promising candidate for that replacement, but time will tell).

Daveed

 

[David Vandevoorde]

@j21g2000yqh.googlegroups.com>, (e-mail address removed) says... [...]

Something better has already been proposed, but not adopted, for
lack of time.  Perhaps if as much effort had been put into it as
was put into getting rid of export.  (Had the proposal been to
replace export with something better, I'd not be opposed.)

I, for one, would certainly _prefer_ a "make before break" change --
but I think it's unrealistic. I'd say a large part of the reason time
wasn't found for Daveed's module proposal was precisely because
export was present

No, that had nothing to do with it.

The only reason (and it's a good reason) "C++ modules" were dropped in
Portland was that it was a big endeavor that was still in its infancy
when we hoped (at that time) to finish up the standard's revision
without two years or so.

-- and I think almost any other such proposal
would have been treated the same was as long as export remained in
the standard.

I don't think that's the case either.

Daveed