realloc equivalent for new

[Michael]

How would I go about shrinking the buffer that was allocated with new, or
expanding it in place? I basically need a realloc equivalent for new.
Thanks in advance.

Michael

 

[Andre Kostur]

How would I go about shrinking the buffer that was allocated with new, or
expanding it in place? I basically need a realloc equivalent for new.
Thanks in advance.

Why? Has your object changed size? And if you're using an array, why
aren't you using vector<> ?

 

[Michael]

Why?

To reduce memory usage.

And if you're using an array, why
aren't you using vector<> ?

I prefer to use new because I'm more familiar with pointers. I did read
about about vector and it doesn't seem to have a way to reduce the size of
the buffer.

 

[Jeff Schwab]

To reduce memory usage.




I prefer to use new because I'm more familiar with pointers. I did read
about about vector and it doesn't seem to have a way to reduce the size of
the buffer.

#include <iostream>
#include <vector>

template< typename Container >
void tight( Container& c )
{
Container d( c.begin( ), c.begin( ) + c.size( ) );
c.swap( d );

std::cout << "size: " << c.size( ) << ", capacity: " <<
c.capacity( )
<< "\n";
}

int main( )
{
std::vector< char > v( 100 );

tighten( v );
v.resize( 30 );
tighten( v );

 

[Ioannis Vranos]

I prefer to use new because I'm more familiar with pointers. I did read
about about vector and it doesn't seem to have a way to reduce the size of
the buffer.

There is no alternative to realloc() simply because it is not needed.
Consider using a vector. An example:

#include <vector>

int main()
{
using namespace std;

//A vector of 10 ints initialised to 0
vector<int> v(10);

// Changed the size to 5 ints
v.resize(5);


// A "2 dimensional" vector - a vector of vectors - with initial size
// 10x2. All ints initialised to 0.
vector<vector<int> >dv(10, vector<int>(2));

// Change to 5x2.
dv.resize(5);
}


By using C++ as only C, makes you missing all the additional benefits of it.


If you already know C, a nice book to read is "Accelerated C++" by
Andrew Koenig, Barbara Moo:

http://www.acceleratedcpp.com

 

[Gianni Mariani]

How would I go about shrinking the buffer that was allocated with new, or
expanding it in place? I basically need a realloc equivalent for new.
Thanks in advance.

There is no realloc equivalent, although, in theory, it would not be
very difficult to write if you could change the malloc interface. What
you would need is a "realloc_no_move" in which realloc would simply do
nothing if your realloc request could not be satisfied in place in which
case the alternative of malloc-copy-free would be performed as an
alternative. There is some discussion about "move" semantics being
added in the next revision of the C++ standard, but I don't know how it
relates to realloc like semantics.

Objects allocated with new must be copied to a new memory location,
using a copy constructor. realloc() simply does a memcpy when the new
allocated memory needs to be moved.

As a comparison, I has written a "buffer" which internally used
vector<char>. This was a class that provided an abstract interface. A
collegue decided that "for performance reasons" would implement a
malloc/realloc/free version and there was a 5-10% performance advantage
using "malloc/realloc/free" over std::vector in running the overall
application (which was pretty heavy on buffer manipulation).

YMMV

 

[Ioannis Vranos]

There is no realloc equivalent, although, in theory, it would not be
very difficult to write if you could change the malloc interface. What
you would need is a "realloc_no_move" in which realloc would simply do
nothing if your realloc request could not be satisfied in place in which
case the alternative of malloc-copy-free would be performed as an
alternative. There is some discussion about "move" semantics being
added in the next revision of the C++ standard, but I don't know how it
relates to realloc like semantics.

malloc() family should be avoided. I think it is easy to make your own
realloc (for new/delete family) version if you want, by using the low
level facilities of C++. For example check allocators and unitialised
memory part, at the end of chapter 19 (I haven't read them thoroughly
though).

Objects allocated with new must be copied to a new memory location,
using a copy constructor. realloc() simply does a memcpy when the new
allocated memory needs to be moved.

As a comparison, I has written a "buffer" which internally used
vector<char>. This was a class that provided an abstract interface. A
collegue decided that "for performance reasons" would implement a
malloc/realloc/free version and there was a 5-10% performance advantage
using "malloc/realloc/free" over std::vector in running the overall
application (which was pretty heavy on buffer manipulation).

Probably you did not use the appropriate container. Check TC++PL 3, 17.1.2.

What was that class about?

 

[Gianni Mariani]

malloc() family should be avoided. I think it is easy to make your own
realloc (for new/delete family) version if you want, by using the low
level facilities of C++.

What facilties might they be ?

For example check allocators and unitialised

memory part, at the end of chapter 19 (I haven't read them thoroughly
though).

As far as I can tell, no standard C++ mechanisms are available for
performing true "realloc" semantics for arrays of any type of C++
objects unless you implement you own malloc-like library. However, if
you implememt your own malloc like library. you have the problem of not
being able to share memory between the two different heaps (the libc
malloc vs the special realloc implementation).

The semantics of realloc require it to "extend" or "shrink" the memory
region and when an "extend" is not possible (because of collision in the
heap with another allocated region) to allocate a larger region and copy
the contents from the original region. This certainly makes the issue
of exceptions an interesting one (what if you throw in a copy
constructor during the copy within realloc++ ?).

Probably you did not use the appropriate container. Check TC++PL 3, 17.1.2.

Why do you think that ?

What was that class about?

An abstract buffer class for reading/writing blobs from various system
utilities. A "buffer" may be implemented in a system specific way (e.g.
memory mapped file) or may be non-specific (plain memory).

What I referred to above (the vector vs malloc performance issue) was
the "plain memory" buffer.

 

[Ioannis Vranos]

What facilties might they be ?

For example check allocators and unitialised

.... of TC++PL 3.

As far as I can tell, no standard C++ mechanisms are available for
performing true "realloc" semantics for arrays of any type of C++
objects unless you implement you own malloc-like library.

I do not know for sure since I haven't learned the entire language yet,
however what are those allocators and unitialised memory facilities about.

However, if
you implememt your own malloc like library. you have the problem of not
being able to share memory between the two different heaps (the libc
malloc vs the special realloc implementation).

The semantics of realloc require it to "extend" or "shrink" the memory
region and when an "extend" is not possible (because of collision in the
heap with another allocated region) to allocate a larger region and copy
the contents from the original region. This certainly makes the issue
of exceptions an interesting one (what if you throw in a copy
constructor during the copy within realloc++ ?).

The whole process implies that you would probably have to overload the
entire family of new/delete (it is allowed to do that). Perhaps some
people in clc++ who are involved in compiler development may shed some
light on this (like Greg Comaeu).


Still, the use of an appropriate container is an easier method to do
what you want for most stuff.

Why do you think that ?



An abstract buffer class for reading/writing blobs from various system
utilities.

What are blobs?

A "buffer" may be implemented in a system specific way (e.g.
memory mapped file) or may be non-specific (plain memory).

What I referred to above (the vector vs malloc performance issue) was
the "plain memory" buffer.

If you use the buffer as a list for example, an std::list would have
better performance.

Each container has its advantages and disadvantages. In sorting for
example, a list is far run-time superior to a vector, since in the case
of the list only pointer values get changed, while in vector entire
objects get copied around.

 

[Ioannis Vranos]

If you use the buffer as a list for example, an std::list would have
better performance.

Each container has its advantages and disadvantages. In sorting for
example, a list is far run-time superior to a vector, since in the case
of the list only pointer values get changed, while in vector entire
objects get copied around.

Also we should never forget vector::reserve() that can be critical on
performance, in case of a vector that has its size continuously changed.

 

[Gianni Mariani]

Ioannis Vranos wrote:
....

I do not know for sure since I haven't learned the entire language yet,
however what are those allocators and unitialised memory facilities about.

I don't have that book, so I don't know what you're referrinf to.

I know that C++ does have "operator new" and "operator delete" which are
very close to the semantics of malloc/free and often are implemented
using malloc and free. Some C++ implementations provide an "optimized"
version of these operators in the hope that they perform better than
plain malloc/free. No realloc semantics however !

The whole process implies that you would probably have to overload the
entire family of new/delete (it is allowed to do that). Perhaps some
people in clc++ who are involved in compiler development may shed some
light on this (like Greg Comaeu).

Greg would say that ther is not C++ equivalent of realloc for generic
array types. While it would be somewhat safe to do this for POD types
(depending on the application of course), there is no general realloc
mechanism. The best candidate for this feature, vector<T>, does not use
any realloc semantics. (As the revision of the standard is today).

....

What are blobs?

Binary Large OBjects

If you use the buffer as a list for example, an std::list would have
better performance.

How would you map a std::list to a mapped file ? A mapped file usually
looks like an array of char. The contents of a vector<char> can also be
made to look like an array of char, i.e. (&v[0]). std::list<char> would
not have an internal structure that resembled anything like an array
of char.

Each container has its advantages and disadvantages. In sorting for
example, a list is far run-time superior to a vector, since in the case
of the list only pointer values get changed, while in vector entire
objects get copied around.

this may or may not be true. It depends on the application. e.g.
sorting a vector<char> would have comparable performance to sorting a
std::list<char>.

 

[Ioannis Vranos]

I don't have that book, so I don't know what you're referrinf to.

Things about class alocator in <memory> that uses operator new() by
default, and how to create your own ones:


"Implementers of containers often allocate() and deallocate() objects
one at a time. For a naive implementation of allocate(), this implies
lots of calls of operator new , and not all implementations of operator
new are efficient when used like that. As an example of a user-defined
allocator, I present a scheme for using pools of fixed-sized pieces of
memory from which the allocator can allocate() more efficiently than can
a conventional and more general operator new().

I happen to have a pool allocator that does approximately the right
thing, but it has the wrong interface (because it was designed years
before allocators were invented). This Pool class implements the notion
of a pool of fixed-sized elements from which a user can do fast
allocations and deallocations. It is a low-level type that deals with
memory directly and worries about alignment:"


Some other facilities:

"19.4.4 Uninitialized Memory

In addition to the standard allocator , the <memory > header provides a
few functions for dealing with uninitialized memory. They share the
dangerous and occasionally essential property of using a type name T to
refer to space sufficient to hold an object of type T rather than to a
properly constructed object of type T .

The library provides three ways to copy values into uninitialized space:"

and it talks about uninitialized_copy, uninitialized_fill and uninitial
ized_fill_n.


Then, other facilities are mentioned like

"Algorithms often require temporary space to perform acceptably. Often,
such temporary space is best allocated in one operation but not
initialized until a particular location is actually needed.

Consequently, the library provides a pair of functions for allocating
and deallocating uninitialized space:

template <class T> pair <T *,ptrdiff_t > get_temporary_buffer(ptrdiff_t
); // allocate, don’t initialize

template <class T> void return_temporary_buffer(T *); // deallocate,
don’t destroy

A get_temporary_buffer<X >(n) operation tries to allocate space for n or
more objects of type X .

If it succeeds in allocating some memory, it returns a pointer to the
first uninitialized space and the number of objects of type X that will
fit into that space; otherwise, the second value of the pair is zero.
The idea is that a system may keep a number of fixed-sized buffers ready
for fast allocation so that requesting space for n objects may yield
space for more than n . It may also yield less, however, so one way of
using get_temporary_buffer() is to optimistically ask for a lot and then
use what happens to be available."


And other stuff, it is an entire chapter.

 

[Rene Moehring]

There is no alternative to realloc() simply because it is not needed.
Consider using a vector. An example:

#include <vector>

int main()
{
using namespace std;

//A vector of 10 ints initialised to 0
vector<int> v(10);

// Changed the size to 5 ints
v.resize(5);


// A "2 dimensional" vector - a vector of vectors - with initial size
// 10x2. All ints initialised to 0.
vector<vector<int> >dv(10, vector<int>(2));

// Change to 5x2.
dv.resize(5);
}

resize cannot reduce the memory consumption. It just says there are x
objects in the vector which has nothing to do with the buffer size that
you set with reserve. But reserve can just increase the buffer size not
decrease.