shared_ptr and incomplete types

Discussion in 'C++' started by mike.polyakov@gmail.com, Nov 28, 2007.

  1. Guest

    I have trouble understanding incomplete types and their interplay with
    shared_ptr. Consider the following code, composed of two source files
    and one header:

    //------------------------------------------------------------------
    // test.h
    #ifndef TEST_H_
    #define TEST_H_
    #include <boost/shared_ptr.hpp>
    using namespace boost;

    struct A;
    struct B
    {
    shared_ptr<A> p;
    B();
    ~B();
    };
    #endif


    //------------------------------------------------------------------
    // test1.cpp
    #include "test.h"
    #include <iostream>
    using namespace std;

    struct A {
    ~A();
    };
    A::~A() { cout <<"Destruct A" <<endl; }

    B::B() : p(new A) {}


    //------------------------------------------------------------------
    // test2.cpp
    using namespace std;
    using namespace boost;

    #include "test.h"

    B::~B() { }

    int main()
    {
    B b;
    return 0;
    }
    //------------------------------------------------------------------


    From my understanding the above should not compile. A is incomplete in
    test2.cpp. Instantiation of shared_ptr<A> destructor should happen
    during compilation of B's destructor. Since B's destructor is declared
    in test2.cpp, the instantiation of shared_ptr<A> destructor should
    cause checked_delete to be applied to incomplete type A and fail.
    However, it doesn't and this compiles and runs correctly. However,
    adding two lines

    A *a;
    shared_ptr<A> p(a);

    to main() in test.cpp generates a compile error, which of course it
    should. I am confused. Could anyone clarify this for me please?
    Thanks.
     
    , Nov 28, 2007
    #1
    1. Advertising

  2. Guest

    I think I have made some progress in understanding this problem. I
    should have read boost documentation more carefully. It is actually
    the constructor of shared_ptr<A>, and not the destructor which
    requires A to be a complete type. This condition is satisfied in the
    above code and hence it compiles. I am guessing that the function with
    'delete' statement is generated during compilation of the constructor.
    During constructor invocation, a pointer to that function is stored
    somewhere inside shared_ptr<A> and its destructor deletes the object
    by calling this function indirectly through that pointer. That way
    shared_ptr<A>'s destructor can appear in translation unit where A is
    an incomplete_type and still be able to delete pointer to A as though
    it new how A was declared. Please correct me if I'm wrong on this.
    Thanks.
     
    , Nov 28, 2007
    #2
    1. Advertising

  3. Kai-Uwe Bux Guest

    wrote:

    > I think I have made some progress in understanding this problem. I
    > should have read boost documentation more carefully. It is actually
    > the constructor of shared_ptr<A>, and not the destructor which
    > requires A to be a complete type. This condition is satisfied in the
    > above code and hence it compiles. I am guessing that the function with
    > 'delete' statement is generated during compilation of the constructor.
    > During constructor invocation, a pointer to that function is stored
    > somewhere inside shared_ptr<A> and its destructor deletes the object
    > by calling this function indirectly through that pointer. That way
    > shared_ptr<A>'s destructor can appear in translation unit where A is
    > an incomplete_type and still be able to delete pointer to A as though
    > it new how A was declared. Please correct me if I'm wrong on this.


    Nope, you pretty much figured it out. The problem of dealing with incomplete
    types is one of the reasons that shared_ptr<> supports a custom deleter.



    Best

    Kai-Uwe Bux
     
    Kai-Uwe Bux, Nov 28, 2007
    #3
  4. wrote:
    > During constructor invocation, a pointer to that function is stored
    > somewhere inside shared_ptr<A>


    That's one thing which I find worrying about shared_ptr. While it's
    nice that you can safely create shared_ptrs of incomplete types, the
    price for this is that shared_ptr becomes quite bulky. If I'm not
    completely mistaken, a shared_ptr object has the size of 3 pointers plus
    it allocates dynamically memory for an integral type, which means that
    it additionally uses memory required by the integral type plus any
    ancillary memory required by a dynamically-allocated object required by
    the memory management and possibly memory alignment. This means that one
    single "pointer" could, depending on the system, require something like
    64 bytes or even more. (Compare this to a system where you have a smart
    pointer which uses a reference counter in the object itself, and
    requires complete types: Only 1 pointer is needed in the smart pointer
    object, and adding the size of the reference counter to it, the total
    memory requirement in a typical 32-bit system is 8 bytes.)

    This can become a major issue if you are, for example, creating an
    array of millions of shared_ptrs. Just the shared_ptrs themselves could
    easily require more memory than the objects they are point to, if the
    objects are small. Since shared_ptr hides this issue very well, a
    typical programmer might not be aware of this.

    And better not copy shared_ptrs in a tight inner loop which requires
    extreme speed...
     
    Juha Nieminen, Nov 28, 2007
    #4
  5. Pete Becker Guest

    On 2007-11-28 09:00:23 -0500, Juha Nieminen <> said:

    > wrote:
    >> During constructor invocation, a pointer to that function is stored
    >> somewhere inside shared_ptr<A>

    >
    > That's one thing which I find worrying about shared_ptr. While it's
    > nice that you can safely create shared_ptrs of incomplete types, the
    > price for this is that shared_ptr becomes quite bulky. If I'm not
    > completely mistaken, a shared_ptr object has the size of 3 pointers plus
    > it allocates dynamically memory for an integral type


    It's typically two pointers. One holds the A* that the shared_ptr<A>
    deals with, and the other holds a pointer to an allocated block that
    contains the pointer that was passed to the constructor, the reference
    count, and the deleter.

    > , which means that
    > it additionally uses memory required by the integral type plus any
    > ancillary memory required by a dynamically-allocated object required by
    > the memory management and possibly memory alignment. This means that one
    > single "pointer" could, depending on the system, require something like
    > 64 bytes or even more. (Compare this to a system where you have a smart
    > pointer which uses a reference counter in the object itself, and
    > requires complete types: Only 1 pointer is needed in the smart pointer
    > object, and adding the size of the reference counter to it, the total
    > memory requirement in a typical 32-bit system is 8 bytes.)
    >
    > This can become a major issue if you are, for example, creating an
    > array of millions of shared_ptrs. Just the shared_ptrs themselves could
    > easily require more memory than the objects they are point to, if the
    > objects are small. Since shared_ptr hides this issue very well, a
    > typical programmer might not be aware of this.


    If so, then a "typical" programmer is incompetent.

    >
    > And better not copy shared_ptrs in a tight inner loop which requires
    > extreme speed...


    Copying shared_ptr objects is fast and cheap. Not as cheap as copying a
    naked pointer, but if you need the semantics of a shared_ptr, the cost
    is not prohibitive. Two pointer copies and an integer increment.

    --
    Pete
    Roundhouse Consulting, Ltd. (www.versatilecoding.com) Author of "The
    Standard C++ Library Extensions: a Tutorial and Reference
    (www.petebecker.com/tr1book)
     
    Pete Becker, Nov 28, 2007
    #5
  6. James Kanze Guest

    On Nov 28, 3:00 pm, Juha Nieminen <> wrote:
    > wrote:
    > > During constructor invocation, a pointer to that function is stored
    > > somewhere inside shared_ptr<A>


    > That's one thing which I find worrying about shared_ptr. While it's
    > nice that you can safely create shared_ptrs of incomplete types, the
    > price for this is that shared_ptr becomes quite bulky. If I'm not
    > completely mistaken, a shared_ptr object has the size of 3 pointers plus
    > it allocates dynamically memory for an integral type, which means that
    > it additionally uses memory required by the integral type plus any
    > ancillary memory required by a dynamically-allocated object required by
    > the memory management and possibly memory alignment.


    It's typically only two pointers, I think. I can't think of any
    reason why there would be a third.

    There are two basic philosophies with regards to reference
    counted pointers: invasive, and non-invasive. Invasive
    reference counted pointers are only a single pointer, require no
    extra allocations, and are significantly safer, in that you can
    create a new reference counted pointer from a raw pointer at any
    time, even if other reference counted pointers already exist.
    (Because they are smaller, they are also slightly faster, but
    this is rarely an issue.) On the other hand, they are invasive;
    the object being pointed to must know about them (typically be
    deriving from a common base class---virtually, if the hierarchy
    is open). Which means no reference counted pointers to existing
    classes, nor to non-class types. For a "standard" pointer,
    that's pretty much a killer exclusion.

    > This means that one single "pointer" could, depending on the
    > system, require something like 64 bytes or even more. (Compare
    > this to a system where you have a smart pointer which uses a
    > reference counter in the object itself, and requires complete
    > types: Only 1 pointer is needed in the smart pointer object,
    > and adding the size of the reference counter to it, the total
    > memory requirement in a typical 32-bit system is 8 bytes.)


    That doesn't match up with my measurements. On my 32-bits
    systems, alignment considerations mean that the counter itself
    ends up requiring 8 bytes, so my invasive reference counted
    pointers require a total of n*4+8 bytes for each object (where n
    is the number of pointers to that object). I think
    boost::shared_ptr requires more, but the one time I implemented
    non-invasive reference counting, I used a custom allocator for
    the ints, with the result that the pointers required n*8+4
    bytes. Not a big difference, unless, of course, you have large
    arrays of reference counted pointers.

    A much more important consideration is safety. Consider a
    simplified example:

    T* p = new T ;
    Ptr< T > p1( p ) ;
    Ptr< T > p2( p ) ;

    With a non-invasive pointer, such as boost::shared_ptr, this
    breaks, resulting the memory being freed prematurely. With
    invasive pointers, it works.

    > This can become a major issue if you are, for example, creating an
    > array of millions of shared_ptrs.


    Yes, but is there ever any reason to have a container of
    shared_ptr?

    > Just the shared_ptrs themselves could
    > easily require more memory than the objects they are point to, if the
    > objects are small. Since shared_ptr hides this issue very well, a
    > typical programmer might not be aware of this.


    That's even true of the invasive pointers. I'd guess that most
    of the time I'm using reference counted pointers, it's for
    "agent" classes, with no data members (except for the implicit
    vptr).

    > And better not copy shared_ptrs in a tight inner loop which requires
    > extreme speed...


    I don't think that the difference will normally be that great.

    The one case there might be a significant difference is if the
    compiler will return (and pass) class types which fit in a
    single register in registers. An invasive pointer will
    typically fit in a single register, a non-invasive one won't.
    But while this optimization has often been discussed, I don't
    know of a single compiler which will do it (unless the class
    type is a POD, but no reference counted pointer will be a POD).
    So it's rather accademic.

    --
    James Kanze (GABI Software) email:
    Conseils en informatique orientée objet/
    Beratung in objektorientierter Datenverarbeitung
    9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
     
    James Kanze, Nov 29, 2007
    #6
  7. Pete Becker Guest

    On 2007-11-29 05:16:00 -0500, James Kanze <> said:

    >
    > That doesn't match up with my measurements. On my 32-bits
    > systems, alignment considerations mean that the counter itself
    > ends up requiring 8 bytes, so my invasive reference counted
    > pointers require a total of n*4+8 bytes for each object (where n
    > is the number of pointers to that object). I think
    > boost::shared_ptr requires more,


    It does, because it does more. std::tr1::shared_ptr<T> traffics in
    T*'s, but the control block holds a copy of the pointer that was passed
    to the constructor, which can point to an object of a type derived from
    T. It also holds an optional deleter, whose type and, therefore, size
    is up to the user.

    --
    Pete
    Roundhouse Consulting, Ltd. (www.versatilecoding.com) Author of "The
    Standard C++ Library Extensions: a Tutorial and Reference
    (www.petebecker.com/tr1book)
     
    Pete Becker, Nov 29, 2007
    #7
  8. James Kanze wrote:
    > On the other hand, they are invasive;
    > the object being pointed to must know about them (typically be
    > deriving from a common base class---virtually, if the hierarchy
    > is open). Which means no reference counted pointers to existing
    > classes, nor to non-class types. For a "standard" pointer,
    > that's pretty much a killer exclusion.


    I wonder if the language couldn't be enhanced so that you can allocate
    existing objects and non-class types in such a way that the compiler
    will internally make the allocation larger by sizeof reference counter
    and then a special internal shared pointer can be used to manage objects
    of this type.

    Perhaps something like:

    shared SomeClass* ptr = shared_new SomeClass();

    That 'ptr' would be of the same size as a regular pointer. The memory
    amount allocated by 'shared_new' would be sizeof(size_t) (or whatever)
    larger than the memory allocated by the equivalent 'new'.

    The semantics could perhaps be so that these would be erroneous:

    shared SomeClass* ptr = new SomeClass(); // error
    SomeClass* ptr = shared_new SomeClass(); // error

    shared SomeClass* ptr = shared_new SomeClass();
    shared SomeClass* ptr2 = ptr; // Ok
    SomeClass* ptr3 = ptr; // error

    SomeClass* ptr = new SomeClass();
    shared SomeClass* ptr2 = ptr; // error

    >> This can become a major issue if you are, for example, creating an
    >> array of millions of shared_ptrs.

    >
    > Yes, but is there ever any reason to have a container of
    > shared_ptr?


    For example if you want a vector containing different types of objects
    (which all have been derived from a common base class) and want the
    memory taken by those objects be managed.
     
    Juha Nieminen, Nov 29, 2007
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Philippe Guglielmetti
    Replies:
    4
    Views:
    924
    tom_usenet
    Oct 9, 2003
  2. aegis

    sizeof and incomplete types

    aegis, Dec 4, 2005, in forum: C Programming
    Replies:
    2
    Views:
    353
    Eric Sosman
    Dec 4, 2005
  3. Alan Johnson
    Replies:
    1
    Views:
    462
    Alf P. Steinbach
    May 13, 2006
  4. Colin Caughie
    Replies:
    1
    Views:
    766
    Shooting
    Aug 29, 2006
  5. Tim H
    Replies:
    3
    Views:
    347
    Marcus Kwok
    Mar 14, 2007
Loading...

Share This Page