Class invariants and implicit move constructors (C++0x)

[Alf P. Steinbach /Usenet]

* SG, on 23.08.2010 10:02:

Howard Hinnant, on 16.08.2010 01:02:

If you would like to officially propose a solution, contact me
privately (with draft in hand) and I will help you get a paper
published.

OK, thanks.
[...]
PS: This is an imperfect draft...

Any updates you would like to share, Alf?

No. Howard stated that many experts on the committee, including Bjarne,
disagreed with something unspecified in this draft. But I think that was his own
interpretation of earlier writings on the issue, not that those people had read
the draft. He recommended I try it on as a defect report instead of as a paper,
but as far as I know there are no defect reports for a draft standard.

I think your suggestion else-thread, about not generating the move ops
automatically when there is a defined assignment op or defined destructor
(unless known that it has empty body), would be the best solution.

It might seem draconian but I don't think it would exclude much code needlessly,
and I think the Really Important aspect here is that moving is an optimization
and that an optimization shouldn't break things but should have the exact same
effect as without the optimization, only faster or less memory.


Cheers,

- Alf

 

[Howard Hinnant]

* SG, on 23.08.2010 10:02:

Howard Hinnant, on 16.08.2010 01:02:
If you would like to officially propose a solution, contact me
privately (with draft in hand) and I will help you get a paper
published.
OK, thanks.
[...]
PS: This is an imperfect draft...

Any updates you would like to share, Alf?

No. Howard stated that many experts on the committee, including Bjarne,
disagreed with something unspecified in this draft. But I think that was his own
interpretation of earlier writings on the issue, not that those people had read
the draft.

My email records indicate that the "something unspecified" referred to
above was not in the draft, but an assertion made by Alf in a private
email to me that the issues of special move functions, and the
exception safety problem described in N2855 were orthogonal.

For whatever reasons (and I'm willing to accept half the blame), Alf
and I were unable to communicate with each other.

He recommended I try it on as a defect report instead of as a paper,
but as far as I know there are no defect reports for a draft standard.

Here is a quote I wrote to Alf.

I can not forward a paper to core without wording. However if you would like to
send your paper in as a core issue, that seems appropriate to me. Here is the
latest core issues list:

http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_active.html

Contact the author of that list and ask if you can submit your paper as a core
issue.

If you would prefer the paper route, I recommend developing wording by working
with members of the CWG who have been working this area. Their names/emails are
on the papers I have been referencing. I should point out that although your
paper does point out a problem, it does not appear to offer a solution to the
problem which these authors have been attempting to address.

If anyone has any better advice for Alf, or can state the above advice
more clearly, I would be obliged.

I can report that the subject was picked up and briefly discussed on
the internal core language mailing list. As far as I can tell, no
conclusions were reached. I do not know if a core issue was opened on
the subject, nor if someone decided to write a paper on it (with
proposed wording). However a mailing deadline has just passed and the
post-Rapperswil mailing will appear here:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/

some time later this week. If an issue was opened, or a paper
written, you'll see it then and there.

-Howard

 

[SG]

SG, on 23.08.2010 10:02:

Any updates you would like to share, Alf?

[snip]

I think your suggestion else-thread, about not generating the move ops
automatically when there is a defined assignment op or defined destructor
(unless known that it has empty body), would be the best solution.

I also included user-declared a copy constructor in that list.
Currently, I like this approach. Though, I can't rule out that there
is something that I'm missing so I welcome other feedback. These are
rules that are just a tad more restrictive than the ones proposed in
N3053. And with these rules both Scott's example and yours would be
fine and not break in C++0x mode.

It might seem draconian but I don't think it would exclude much code needlessly,
and I think the Really Important aspect here is that moving is an optimization
and that an optimization shouldn't break things but should have the exact same
effect as without the optimization, only faster or less memory.

Right.

My email records indicate that the "something unspecified" referred to
above was not in the draft, but an assertion made by Alf in a private
email to me that the issues of special move functions, and the
exception safety problem described in N2855 were orthogonal.

This is how I see it. Please correct any mistakes:

N2855:
points out that throwing move constructors (without any additional
compiler-magic to cope with this) poses a problem in terms of
exception safety. Example: class template pair. Depending on the
template parameters it might or might not be a good idea to declare
a move constructor -- that is, if you want to avoid throwing move
constructors. Its proposed solution relies on "concepts", or more
specifically: a requires clause to conditionally support move ops
depending on the template parameters.

N2904:
operates also under the viewpoint that throwing move ops are bad.
The idea is to promote move ops to special functions and let the
compiler figure whether it is "safe" to generate them -- always
having aggregate-like types like pair<A,B> with no invariants
in mind which in my humble opinion is a dangerous way of thinking
here.

N3053:
Refinement of N2904. Slight rule changes.

N3050:
refinement of N2855. Now, throwing move ctors are deemed to be okay
if we get a little more compiler magic accessible via the noexcept
operator and the nothrow_xxx traits. With this in our hands we are
able to solve the exception safety issue by either suppressing
move ops via SFINAE and/or by allowing throwing moves by marking
them with noexcept(false) so that std::move_if_noexcept falls back
on a copy if we need the strong exception safety guarantee.

But we're left with possibly dangerous rules of N3053 that lead also
to implicitly generated move ops in cases where we DON'T have simple
aggregate-like types and where old code can break in C++0x mode.

If the rules are changed to be a little more restrictive like I
suggested in the August 18th post we get rid of most breaking examples
of old code (including Scott's and Alf's code), still have implicitly
generated move ops for many aggregate-like types and can solve the
problem pointed out by N2855 with the noexcept exception specification
and the traits we got since N3050. Since I proposed to disable
compiler-generated move ops in case any special functions are user-
declared and pair/tuple actually need to at least have user-defined
assignment operators so that "reference members" are correctly dealt
with, we would have to write a pair/tuple class template with user-
defined move ops like this:

template<class T, class U>
struct pair {
T first;
U second;

...

pair(pair && p) noexcept(
nothrow_constructible<T,T&&>::value
&& nothrow_constructible<U,U&&>::value )
: first (forward<T>(p.first))
, second(forward<U>(p.second))
{}

pair& operator=(pair && p) noexcept(
noexcept(first = forward<T>(p.first ))
&& noexcept(second = forward<U>(p.second)) )
{
first = forward<T>(p.first);
second = forward<U>(p.second);
}

...
};

Of course, if you don't want the move ops at all in cases they are not
exception-safe, we really need something like a requires-clause. I'd
be happy to have one without the whole concepts machinery, actually.
Just a requires-clause that accepts a compile-time bool constant that
depends on some template parameters. It doesn't give us "modular type
checking" in the sense that Doug Gregor explained it in one of his
concepts talks but it would be a newbie-friendly SFINAE replacement
and could be used on non-template members of class templates as well.
But I digress...

I can report that the subject was picked up and briefly discussed on
the internal core language mailing list. As far as I can tell, no
conclusions were reached. I do not know if a core issue was opened on
the subject, nor if someone decided to write a paper on it (with
proposed wording). However a mailing deadline has just passed and the
post-Rapperswil mailing will appear here:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/

some time later this week. If an issue was opened, or a paper
written, you'll see it then and there.

Thanks for the update, Howard! I'm glad to hear that it as been
discussed at all.

Cheers!
SG

 

[SG]

[...]
My email records indicate that the "something unspecified" referred to
above was not in the draft, but an assertion made by Alf in a private
email to me that the issues of special move functions, and the
exception safety problem described in N2855 were orthogonal.

This is how I see it. Please correct any mistakes:

N2855:
  points out that throwing move constructors (without any additional
  compiler-magic to cope with this) poses a problem in terms of
  exception safety. Example: class template pair. Depending on the
  template parameters it might or might not be a good idea to declare
  a move constructor -- that is, if you want to avoid throwing move
  constructors. Its proposed solution relies on "concepts", or more
  specifically: a requires clause to conditionally support move ops
  depending on the template parameters.

N2904:
  operates also under the viewpoint that throwing move ops are bad.
  The idea is to promote move ops to special functions and let the
  compiler figure whether it is "safe" to generate them -- always
  having aggregate-like types like pair<A,B> with no invariants
  in mind which in my humble opinion is a dangerous way of thinking
  here.

N3053:
  Refinement of N2904. Slight rule changes.

N3050:
  refinement of N2855. Now, throwing move ctors are deemed to be okay
  if we get a little more compiler magic accessible via the noexcept
  operator and the nothrow_xxx traits. With this in our hands we are
  able to solve the exception safety issue by either suppressing
  move ops via SFINAE and/or by allowing throwing moves by marking
  them with noexcept(false) so that std::move_if_noexcept falls back
  on a copy if we need the strong exception safety guarantee.

But we're left with possibly dangerous rules of N3053 that lead also
to implicitly generated move ops in cases where we DON'T have simple
aggregate-like types and where old code can break in C++0x mode.

If the rules are changed to be a little more restrictive like I
suggested in the August 18th post we get rid of most breaking examples
of old code (including Scott's and Alf's code), still have implicitly
generated move ops for many aggregate-like types and can solve the
problem pointed out by N2855 with the noexcept exception specification
and the traits we got since N3050.

I would like to point out this has been addressed by two (competing)
proposals:

[1] N3153: "Implicit Move Must Go" (D. Abrahams)
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3153.htm

This is about disabling any implicit generation of move operations but
still allowing the =default syntax to avoid any C++98 compatibility
issues and possible, unanticipated traps ("so late in the game").

[2] N3174: "To move or not to move" (B. Stroustrup)
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3174.pdf

This proposal is basically similar to what we came up with here (see
my August 18th post). User-declared copy/move functions (including the
=default and =delete syntax) as well as a user-declared destructor
indicate possible invariants and the idea is to suppress compiler-
generated copy/move operations altogether in this case. That is /
including/ implicit generation of copy. Yes, you read correctly.
Stroustrup is actually proposing to /deprecate/ implicit copy ctors
and copy assignments in these cases as well. And "deprecate" means,
it's still generated but it would be nice if a compiler warned about
it in C++0x mode. This way, the rules for implicit generation of copy
and move are similar, easy to remember and we get rid of many "rule of
three" traps, beginners like to step into...

Regarding Dave's std::remove example (see
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3153.htm ),
the rules of N3174 still might surprize a user. But we could also
argue that std::remove is not backwards compatible because it requests
the elements to move instead of copying them. We could also argue that
the user wrote buggy code if he/she really tries to "use" the objects
behind the end of the resulting range.

Cheers!
SG

 

[Howard Hinnant]

Regarding Dave's std::remove example (seehttp://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3153.htm),
the rules of N3174 still might surprize a user. But we could also
argue that std::remove is not backwards compatible because it requests
the elements to move instead of copying them. We could also argue that
the user wrote buggy code if he/she really tries to "use" the objects
behind the end of the resulting range.

Imho C++98/03 specifies that the values behind the end of the
resulting range are valid but unspecified. Furthermore there are
highly motivating reasons for that position. I believe the following
is a valid C++03 optimization of remove:

pair<vector<int>, int>*
remove(pair<vector<int>, int>* __first, pair<vector<int>, int>*
__last,
const pair<vector<int>, int>& __value)
{
__first = std::find(__first, __last, __value);
if (__first != __last)
{
pair<vector<int>, int>* __i = __first;
while (++__i != __last)
{
if (!(*__i == __value))
{
swap(__first->first, __i->first);
__first->second = __i->second;
++__first;
}
}
}
return __first;
}

This works the same as the general remove algorithm in the range
[first, returned-iterator), but gives different results in the range
[returned-iterator, last). And it will execute O(N) faster. Clients
of such an implementation could use the values in [returned-iterator,
last) as long as that use did not assume what the precise values were
upon returning from remove. For example client code could inspect
each pair<vector<int>, int>, call clear() on pair.first, etc.

Defining a moved-from value in C++0X as a valid but unspecified value,
and defining remove to use move, is a backwards compatible change: In
both standards the client sees valid but unspecified values in the
range [returned-iterator, last).

-Howard

 

[SG]

Regarding Dave's std::remove example (see
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3153.htm ),
the rules of N3174 still might surprize a user. But we could also
argue that std::remove is not backwards compatible because it requests
the elements to move instead of copying them. We could also argue that
the user wrote buggy code if he/she really tries to "use" the objects
behind the end of the resulting range.

Imho C++98/03 specifies that the values behind the end of the
resulting range are valid but unspecified.  Furthermore there are
highly motivating reasons for that position.  I believe the following
is a valid C++03 optimization of remove:
[...]
Defining a moved-from value in C++0X as a valid but unspecified value,
and defining remove to use move, is a backwards compatible change:  In
both standards the client sees valid but unspecified values in the
range [returned-iterator, last).

In other words, you're saying that Dave's code examples are
questionable because objects with unspecified values are
"used" (before they are reassigned or destructed). I agree. Not that
it makes any difference, but I'm currently in favor of Stroustrup's
proposal -- including the part about deprecating compiler-generated
copy operations in some cases.

Cheers!
SG

 

[Howard Hinnant]

On Oct 21, 12:26 pm, SG wrote:

Imho C++98/03 specifies that the values behind the end of the
resulting range are valid but unspecified.  Furthermore there are
highly motivating reasons for that position.  I believe the following
is a valid C++03 optimization of remove:
  [...]
Defining a moved-from value in C++0X as a valid but unspecified value,
and defining remove to use move, is a backwards compatible change:  In
both standards the client sees valid but unspecified values in the
range [returned-iterator, last).

In other words, you're saying that Dave's code examples are
questionable because objects with unspecified values are
"used" (before they are reassigned or destructed). I agree. Not that
it makes any difference, but I'm currently in favor of Stroustrup's
proposal -- including the part about deprecating compiler-generated
copy operations in some cases.

Actually no, Dave's example is correct. It is ok to use an
unspecified value as long as you do so in a way that does not violate
the type's invariants. Since you don't know the value of an
unspecified value, then the only things that you can do with such a
value are those things that do not have any preconditions on the
value's state.

For example: If you have a std::vector with an unspecified value, it
is ok to call clear() on it. clear() has no preconditions. But it is
not ok to call pop_back() on it. The vector might be empty, and thus
you would be violating a precondition of pop_back.

In Dave's example the C++03 version of Y has an invariant that
values.size() == 1. Even a Y with unspecified value has values.size()
== 1. In C++03, you should be able to index a Y with 0, even if the
value of Y is unspecified. Dave's argument is that if we give Y
implicit move members, those move members would leave a Y in a state
such that its invariant does not hold (values.size() == 0). And since
std::remove will expose such moved from members, it is possible for
this valid C++03 code to break under C++0X if Y gets implicit move
members. Dave's example is correct.

My post asserts that a C++0X move-based std::remove is backwards
compatible. However that assertion assumes that we do not break a
type's invariants. If we do break a type's invariants, then
std::remove isn't backwards compatible, and neither is any other code
that might expose valid but unspecified values.

For example if vector::insert-in-the-middle throws an exception, or if
std::sort throws an exception, moved-from values are again exposed.

I am not trying to argue either side of the "implicit move members"
debate. I'm trying to describe the role of move in algorithms, and
how it is a backwards compatible change as long as a moved-from value
is valid, though unspecified, which means that all invariants still
have to be intact.

I believe Dave's paper is correct: If we implicitly generate move, we
will have the potential for breaking code. Dave's paper aptly
demonstrates that implicit move comes with a cost, no matter what the
rules are for implicit move generation.

I also believe Bjarne's paper raises a valid point: Perhaps the
benefit outweighs the cost.

-Howard

 

[Patrik Kahari]

Hi

Looking at Scott Meyers example, reading the comments here. and
thinking about move only objects or expensive to copy objects (like
mutex wrappers, framework singletons, etc). The following occurred to
me.

Are not all these proposals workarounds for the fact that the compiler
is calling the destructor for moved from (zombie) objects? And why do
we need the destructor to be called for such objects? It seems to me
all these problems would disappear if the destructor was not called
for moved from objects. It also seems to me to be the correct thing,
from an ownership point of a view. My reasoning below ..

The two following point are not obvious to me.

1) Why are destructor's called for moved from objects?

It seems fundamentally wrong to me. A move should transfers ownership
of one objects internal resources and the responsibilities of its
invariants from one object to another. Its the responsibility of the
destination objects destructor to tear down the resource built up in
the sources constructor. The source should not linger on. Why should
the source object destructor do anything, when it no longer own any of
its previous resources or has any of its invariant responsibilities?

A parallel is how today a destructor's does not clean up an object
that fails halfway through construction. Such objects never properly
came "alive" as they never got full ownership of their resources or
could not reach their invariants. Similarly a moved from object are no
longer "alive" as they no longer own the resources and the invariants
might be broken.

2) Why is required that moved from objects be in a invariant unbroken
state?

Isn't the point of move constructor, to transfer ownership of an
internal resource and its invariant from one object to another? If we
requiring the source object to keep an invariant after the move. That
will mean that at the minimum some empty state re-construction has to
happen for the source. But not all object have an valid empty state.
Default construction might be an valid empty state. But that might
also be very expensive (boost::array). And not all objects have
default constructors anyway. Also requiring the moved object to keep
an invariant, means that there will have to be two valid object after
a move. One equal to the source before the copy, and another one in
some "valid" default or empty state. I think its better for copy to
copy objects, and for move to just move objects, and thereby
invalidating the source.

3) How can we requiring empty state for objects that don't have one?
If the user is to reuse the moved from object afterwards. Any invalid
empty state will mean the user has to do reinitialize it (by
assignment operator) to some valid value before using it. Why not make
just make it undefined behavior for a user to use a moved from object
without re-initializing?

A parallel would be how a user can copy construct one pointer from
another, but before reusing the source pointer for something else the
user has to reinitialize it. Or reusing a pointer after a delete.

Hope someone can answer these question of mine or correct me on any
misunderstandings.

Cheers, Patrik

 

[Alf P. Steinbach /Usenet]

Patrik: I think this is a good idea that deserves good discussion. Disclaimer: I
have the flu so not thinking clearly, but still. So, could you please post this
as a new thread, not buried deep within an old one? Thanks, -Alf

* Patrik Kahari, on 31.10.2010 00:35:

 

[Bo Persson]

Hi

Looking at Scott Meyers example, reading the comments here. and
thinking about move only objects or expensive to copy objects (like
mutex wrappers, framework singletons, etc). The following occurred
to me.

Are not all these proposals workarounds for the fact that the
compiler is calling the destructor for moved from (zombie) objects?
And why do we need the destructor to be called for such objects? It
seems to me all these problems would disappear if the destructor
was not called for moved from objects. It also seems to me to be
the correct thing, from an ownership point of a view. My reasoning
below ..

The following point are not obvious to me.

1) Why are destructor's called for moved from objects?

Destructors are called for all live objects when they go out of scope
(or are explicitly destroyed).

2) Why is required that moved from objects be in a invariant
unbroken state?

The moved from object might be part of a larger object, like a
container. It is generally important that this larger object remains
consistent, even if parts of it are moved somewhere else.

3) How can we requiring empty state for objects that don't have one?

We don't. Moving is an opportunity for a possible optimization. If it
cannot be done properly, or turns out to be more expensive than
copying, then we don't have to use it. Just continue copying, like
before.



Bo Persson

 

[Howard Hinnant]

1) Why are destructor's called for moved from objects?

Because this is not "destructive move semantics". Destructive move
semantics was considered from the beginning. Here is where this
alternative was discussed in 2002:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2002/n1377.htm#Alternative move designs

This discussion lays out some of the problems with the design, and
invites others to work on it. It can coexist with non-destructive
move semantics. Both forms are useful.

Non-destructive move semantics is most useful for sequence permutation
algorithms such as swap or sort. In such an algorithm, the algorithm
ends with the same number of resources it begins with. Just
everything has been *moved* around. Thus destructing every time you
move from something is counterproductive.

2) Why is required that moved from objects be in a invariant unbroken
state?

So that the object can be reused or destructed.

3) How can we requiring empty state for objects that don't have one?

We can't. Not every type has to have a move constructor or move
assignment operator.

-Howard

 

[Chris Cochran]

In this and other discussions about move constructors, there seems to be little attention put on the concept of temporary objects. Move constructors only operate on temporary objects, because if it's not temporary, i.e. has further uses, you better not steal its value away. But if you can determine that an object is temporary, you can move it to make copies, or modify it in-place without concern.

Given a class String, we can derive a temporary version of it, TString, that is used for all "temporary" String objects. TStrings are copied with a move, assigned to a String with a move, and modified in-place as needed.

Temporary objects arise from only one place: functions returning objects by-value. By changing every String fn( ) definition to TString fn( ), moves will be used for all return values. As temps are combined in expressions, temporary results continue to propagate up to the final result, also a temporary.

This approach has worked flawlessly for many years, as applied to strings and numeric arrays of all types in my framework, across various compilers, including MSVC6 and Intel 11.1. It is combined with a method to detect nonheap Strings that are logically temporary, to treat them as TStrings implicitly inside constructors. Single-stepping through expression evaluation shows arbitrary expressions making optimal choices between copy and move in all scenerios. It is thread-safe, exception-safe and compiler friendly.

Move constructors are an implementation artifact, but the concept itself is all about temporary objects and the special powers they possess. A focus on the nature of temporary objects greatly simplifies understanding this amazing optimization.