Length of C++ arrays allocated by operator new[]

[Marcel Müller]

AFAIK it is impossible to detect the original length of an array from
the object pointer.

class SomeClass
{ // ...
};

SomeClass* array = new SomeClass[10];

size_t length = get_length_of_array(array); // Impossible

But the runtime needs to keep track of the number of elements anyway, to
call the appropriate number of destructors at delete[]. Why does the
language not allow to access this internal field?


Marcel

 

[Richard Damon]

AFAIK it is impossible to detect the original length of an array from
the object pointer.

class SomeClass
{ // ...
};

SomeClass* array = new SomeClass[10];

size_t length = get_length_of_array(array); // Impossible

But the runtime needs to keep track of the number of elements anyway, to
call the appropriate number of destructors at delete[]. Why does the
language not allow to access this internal field?


Marcel

Because array could also have been set by:

class SomeClass
{ //
};

SomeClass element;

SomeClass* array = &element;


or you could have even changed you example to have been

SomeClass* array = (new SomeClass[10])[4];


In either of these cases there is no "length" to get.

Also, the runtime isn't always required to keep track of how many
elements are in the array, if SomeClass has a trivial destructor (like
all of its members are base types and no explicit destructor is defined)
than by the "as-if" rule, the run time doesn't need to know or store the
number of elements (new probably still needs to somewhere store the
allocation size for delete, but that is a different issue).

Allowing this help minimize the overhead in things like new char[10]

 

[Paul]

AFAIK it is impossible to detect the original length of an array from
the object pointer.

class SomeClass
{ // ...

};

SomeClass* array = new SomeClass[10];

size_t length = get_length_of_array(array); // Impossible

But the runtime needs to keep track of the number of elements anyway, to
call the appropriate number of destructors at delete[]. Why does the
language not allow to access this internal field?

You could cast the return from new into a pointer of type... Someclass
(*p)[10].

HTH.

 

[A]

SomeClass* array = new SomeClass[10];

size_t length = get_length_of_array(array); // Impossible

any particular reason you don't want to use vector instead and then use
vect::size ?

 

[Paul]

AFAIK it is impossible to detect the original length of an array from
the object pointer.

class SomeClass
{ // ...

};

SomeClass* array = new SomeClass[10];

size_t length = get_length_of_array(array); // Impossible

But the runtime needs to keep track of the number of elements anyway, to
call the appropriate number of destructors at delete[]. Why does the
language not allow to access this internal field?

You could cast the return from new into a pointer of type... Someclass
(*p)[10].

An example:

struct Someclass{};

template<typename T, size_t N>
void foo(T(&t)[N] ){
std::cout<<"The array is of size "<<N<<std::endl;
}

int main() {
Someclass(*p)[10] = reinterpret_cast<Someclass(*)[10]>(new
Someclass[10]);
foo(*p);
//p needs to be dereferenced.

//Create a ref and no need to dereference, see below.
Someclass(&r)[10] = *p;
foo(r);

delete[] p;
}

 

[Juha Nieminen]

AFAIK it is impossible to detect the original length of an array from
the object pointer.

class SomeClass
{ // ...

};

SomeClass* array = new SomeClass[10];

size_t length = get_length_of_array(array); // Impossible

But the runtime needs to keep track of the number of elements anyway, to
call the appropriate number of destructors at delete[]. Why does the
language not allow to access this internal field?

You could cast the return from new into a pointer of type... Someclass
(*p)[10].

Exactly how does that help determining how many elements there are in
a dynamically allocated array?

 

[Paul]

AFAIK it is impossible to detect the original length of an array from
the object pointer.
class SomeClass
{ // ...
};
SomeClass* array = new SomeClass[10];
size_t length = get_length_of_array(array); // Impossible
But the runtime needs to keep track of the number of elements anyway, to
call the appropriate number of destructors at delete[]. Why does the
language not allow to access this internal field?

You could cast the return from new into a pointer of type... Someclass
(*p)[10].

  Exactly how does that help determining how many elements there are in
a dynamically allocated array?- Hide quoted text -

See the example I added.

 

[Richard]

[Please do not mail me a copy of your followup]

Juha Nieminen <[email protected]> spake the secret code

Exactly how does that help determining how many elements there are in
a dynamically allocated array?

If the number of elements in the array is known at compile time, the
size of the array can then be known at compile time as long as the
size of the array is included in the type signature for a pointer to
the elements.

If the number of elements allocated is determined at runtime, then the
type signature can't know how many elements are in the array and
template tricks can't determine it either.

If you have fixed sized arrays that are dynamically allocated, then
you can use the type signature/template trick to determine the number
of elements.

If you have variable sized arrays that are dynamically allocated, then
use std::vector and access the element count with the size member.

 

[Marcel Müller]

But the runtime needs to keep track of the number of elements anyway, to
call the appropriate number of destructors at delete[]. Why does the
language not allow to access this internal field?

Because array could also have been set by:

class SomeClass
{ //
};

SomeClass element;

SomeClass* array = &element;


or you could have even changed you example to have been

SomeClass* array = (new SomeClass[10])[4];


In either of these cases there is no "length" to get.

Fine. But requesting the length of all these objects is no more
undefined behavior than applying delete[] to them. I only expect a
reasonable length information if I could use operator delete[]. In all
other cases it would be obviously a serious overhead.

Also, the runtime isn't always required to keep track of how many
elements are in the array, if SomeClass has a trivial destructor (like
all of its members are base types and no explicit destructor is defined)
than by the "as-if" rule, the run time doesn't need to know or store the
number of elements (new probably still needs to somewhere store the
allocation size for delete, but that is a different issue).

I am in doubt if any recent implementation uses anything else than the
allocation size for the invocation of the destructors.

Allowing this help minimize the overhead in things like new char[10]

That's the point. I see no additional overhead.


Marcel

 

[Marcel Müller]

SomeClass* array = new SomeClass[10];

size_t length = get_length_of_array(array); // Impossible

any particular reason you don't want to use vector instead and then use
vect::size ?

- The old platform does not support STL well.
- STL causes the executable size to explode. One disadvantage of
template meta programming over generics. (Of course, there are many
advantages on the other side.)

Yes, well said. std::vector should be preferred over dynamic arrays for
most cases;

std::vector has the major disadvantage that any piece of code that has
write access to its elements can also change its size and, more
importantly, cause reallocations. That's sometimes not wanted and could
cause hard to find bugs with UB, especially if the array is shared
between threads.

about the only case where one still needs to use dynamic
arrays is if you want to allocate but not initialize a buffer.

Why? vector will always initialize /less/ or same than operator new[].
The allocation size of the vector may be larger than logical size,
containing additional uninitialized storage. new[] in contrast always
invokes the standard constructors (if any).


Marcel

 

[Ian Collins]

SomeClass* array = new SomeClass[10];

size_t length = get_length_of_array(array); // Impossible

any particular reason you don't want to use vector instead and then use
vect::size ?

- The old platform does not support STL well.
- STL causes the executable size to explode.

Only if you have a very poor linker.

One disadvantage of
template meta programming over generics. (Of course, there are many
advantages on the other side.)

Who mentioned template meta-programming?

std::vector has the major disadvantage that any piece of code that has
write access to its elements can also change its size and, more
importantly, cause reallocations. That's sometimes not wanted and could
cause hard to find bugs with UB, especially if the array is shared
between threads.

Then derived privately and only expose a subset of members. Or simply
write your own array class. Either way, the "avoid naked arrays" rule
of thumb is still valid.

about the only case where one still needs to use dynamic
arrays is if you want to allocate but not initialize a buffer.

Why? vector will always initialize /less/ or same than operator new[].
The allocation size of the vector may be larger than logical size,
containing additional uninitialized storage. new[] in contrast always
invokes the standard constructors (if any).

So does vector's sized constructor.

 

[Marcel Müller]

I don't care about old platforms unless I am being paid to care about
old platforms.



Template code bloat is mostly over stated; non-template classes with
lots of inline functions can have similar "bloat".

That's not what I mean. The most bloat comes from different
instantiations of template classes that in fact produce binary the same
code. E.g. std::vector will most likely result in identical code for all
pointer types (except for member function pointers, of course).

At least I have not seen a compiler that can share code between
different template instantiations.

If you are worried about clients changing a vector's size then don't
expose the vector by, for example, using private inheritance (as Ian
said in his reply) or composition.

I ended up with my own class, which is trivial for arrays that are not
resizable.

about the only case where one still needs to use dynamic
arrays is if you want to allocate but not initialize a buffer.

Why? vector will always initialize /less/ or same than operator new[].
The allocation size of the vector may be larger than logical size,
containing additional uninitialized storage. new[] in contrast always
invokes the standard constructors (if any).

Wrong; for POD and built-in types new[] can allocate storage but not
initialize it at all; whilst a vector with N elements will have
initialized them all:

So vector<char> initializes the storage? To zero I guess. I was not
aware of that.


Marcel

 

[Ian Collins]

That is the price you pay for not having to lose type information
however you can have a std::vector<void*> if you want. Template bloat
is less of a concern these days due to increased memory and storage
capacities even on smaller devices such as mobile phones.

You don't even gave to loose type safety if you specialise vector for
void* and derive privately from there.

 

[Ian Collins]

Indeed but pointers are not always the answer; the convenience and
possible performance characteristics of using a container as a container
of value type tend to outweigh concerns of code bloat these days IMHO
(assuming I am not writing code for a £20 toaster).

The £20 toaster is an interesting example. At least one embedded tool
chain I've used (Green Hills) when optimising for space will fold
duplicate machine instructions into calls Thus all vectors of pointer
type will generate the same machine code which gets merged. So the
specialisation trick is unnecessary!

 

[Paul]

On 21/08/2011 23:01, Marcel Müller wrote:

So vector<char> initializes the storage? To zero I guess. I was not
aware of that.

It initializes *elements* in that storage:

std::vector<char> v(100); // will have 100 chars of value 0
char* a = new char[100]; // will have 100 uninitialized chars

You have previoulsy argued that 'a' points to only one single char.
Are you now suggesting that 'a' points to 100 chars?

 

[Paul]

So vector<char>  initializes the storage? To zero I guess. I was not
aware of that.
It initializes *elements* in that storage:
std::vector<char>  v(100); // will have 100 chars of value 0
char* a = new char[100]; // will have 100 uninitialized chars

You have previoulsy argued that 'a' points to only one single char.
Are you now suggesting that 'a' points to 100 chars?

"will have" does not mean "points to"; 'a' points to the first of 100
uninitialized chars; 'a' points into an array of 100 uninitialized chars.

Also in the old thread on this topic I did say that I found the relaxed
(i.e. not technically accurate) language of "points to an array"
acceptable but not "pointer to an array".

With a sequence you have a start point, an end point and all the rest
inbetween. The term "pointing into an array" suggests that it's
pointing to somewhere inbetween the start and end points(non
inclusive).

I think your terminology is confusing what seems to be a simple
contextual difference in the term "is a pointer to" where it can mean
either:
"is a pointer to X memory" or "is a pointer of type pointer to T".

'a' points to the allocated memory. The allocated memory is freed by:
delete[] a;
I think it's techincally inaccurate to suggest that 'a' doesn't point
to the allocated memory(which is an array of chars).

 

[Juha Nieminen]

- STL causes the executable size to explode. One disadvantage of
template meta programming over generics.

Why is this urban legend so persistent? People repeat it without actually
trying it.

I just tested with these two programs:

//---------------------------------------------------------
int main()
{
int* array = new int[10000];
for(int i = 0; i < 10000; ++i) array = i;
int value = 0;
for(int i = 0; i < 10000; ++i) value += array;
delete[] array;
return value;
}
//---------------------------------------------------------

//---------------------------------------------------------
#include <vector>

int main()
{
std::vector<int> array(10000);
for(int i = 0; i < 10000; ++i) array = i;
int value = 0;
for(int i = 0; i < 10000; ++i) value += array;
return value;
}
//---------------------------------------------------------

The size of the first executable? 5104 bytes.
The size of the second executable? 5528 bytes.

Yeah, the size really exploded.

 

[Juha Nieminen]

Template bloat
is less of a concern these days due to increased memory and storage
capacities even on smaller devices such as mobile phones.

What "template bloat"? Why do people keep repeating this mantra as
if it was a self-evident given?

Perhaps people get confused by having compiled programs in debug mode,
with the compiler adding debug information into the executable. Template
code tends to produce significantly longer names than basic types.

However, you don't want to compile a release version with debug info
in the executable, especially on an embedded system.

 

[Marcel Müller]

- STL causes the executable size to explode. One disadvantage of
template meta programming over generics.

Why is this urban legend so persistent? People repeat it without actually
trying it.

I just tested with these two programs:

//---------------------------------------------------------
int main()
{
int* array = new int[10000];
for(int i = 0; i < 10000; ++i) array = i;
int value = 0;
for(int i = 0; i < 10000; ++i) value += array;
delete[] array;
return value;
}
//---------------------------------------------------------

//---------------------------------------------------------
#include <vector>

int main()
{
std::vector<int> array(10000);
for(int i = 0; i < 10000; ++i) array = i;
int value = 0;
for(int i = 0; i < 10000; ++i) value += array;
return value;
}
//---------------------------------------------------------

The size of the first executable? 5104 bytes.
The size of the second executable? 5528 bytes.

Yeah, the size really exploded.

Use it for some dozen different types, not just int.
Use many functions of the STL containers not just the easiest ones.

Templates have to be instantiated for each unique parameter set. Even if
90% of the generated functions do not depend on all parameters or the
dependency ends once the compiler has done the type checks, the compiler
will generate distinct implementations. No programmer would repeat his
self that much without the templates (well, hopefully).


Marcel

 

[Nick Keighley]

On 20/08/2011 17:26, A wrote:
SomeClass* array = new SomeClass[10];
size_t length = get_length_of_array(array); // Impossible
any particular reason you don't want to use vector instead and then use
vect::size ?

- The old platform does not support STL well.

I don't care about old platforms unless I am being paid to care about
old platforms.

- STL causes the executable size to explode. One disadvantage of
template meta programming over generics. (Of course, there are many
advantages on the other side.)

Template code bloat is mostly over stated; non-template classes with
lots of inline functions can have similar "bloat".

std::vector has the major disadvantage that any piece of code that has
write access to its elements can also change its size and, more
importantly, cause reallocations. That's sometimes not wanted and could
cause hard to find bugs with UB, especially if the array is shared
between threads.

If you are worried about clients changing a vector's size then don't
expose the vector by, for example, using private inheritance (as Ian
said in his reply) or composition.  Reallocation behaviour of vectors is
well known; don't blame the tools for sloppy programming.

or use boost array?

Why? vector will always initialize /less/ or same than operator new[].
The allocation size of the vector may be larger than logical size,
containing additional uninitialized storage. new[] in contrast always
invokes the standard constructors (if any).

Wrong; for POD and built-in types new[] can allocate storage but not
initialize it at all; whilst a vector with N elements will have
initialized them all: don't confuse vector::reserve with vector::resize.
  You can not write directly to the reserved but unused (uninitialised)
storage of a vector; you can only write to vector::size() elements.

/Leigh