count and count_if algorithms return type seem to insufficient forlarge values

[subramanian100in]

Below is my understanding about count algorithms.

Return type of count and count_if algorithms is
iterator_traits<InputIterator>::difference_type.

If the container contains more than 'difference_type' elements
satisfying the condition, then count and count_if algorithm cannot
return a value greater than 'difference_type'.

As an example, suppose maximum value of 'difference_type' is INT_MAX.
Then the corresponding maximum value of 'size_type' is UINT_MAX. Now
consider the declaration
vector<char> v;
where 'v' contains UINT_MAX elements each of which is 'a'.

Now if I do
count(v.begin(), v.end(), 'a')
then the actual count would be UINT_MAX but it cannot be represented
in 'difference_type' which is the return type of count algorithm.

So' won't we get erroneous results from count algorithms ?

I understand that difference_type is same as ptrdiff_t for
pointers(belonging to the same array) and size_type if same as size_t.
This means that the count algorithms can never return values from
ptrdiff_t + 1 upto size_t.

Is this understanding correct ?

If what I have mentioned is really a problem with count algorithms,
how should I count the elements in a container satisfying the
condition for large number of elements(ie greater than
difference_type) ?

Kindly clarify.

Thanks
V.Subramanian

 

[Daniel Pitts]

Below is my understanding about count algorithms.

Return type of count and count_if algorithms is
iterator_traits<InputIterator>::difference_type.

If the container contains more than 'difference_type' elements
satisfying the condition, then count and count_if algorithm cannot
return a value greater than 'difference_type'.

As an example, suppose maximum value of 'difference_type' is INT_MAX.
Then the corresponding maximum value of 'size_type' is UINT_MAX. Now
consider the declaration
vector<char> v;
where 'v' contains UINT_MAX elements each of which is 'a'.

Now if I do
count(v.begin(), v.end(), 'a')
then the actual count would be UINT_MAX but it cannot be represented
in 'difference_type' which is the return type of count algorithm.

So' won't we get erroneous results from count algorithms ?

I understand that difference_type is same as ptrdiff_t for
pointers(belonging to the same array) and size_type if same as size_t.
This means that the count algorithms can never return values from
ptrdiff_t + 1 upto size_t.

Is this understanding correct ?

If what I have mentioned is really a problem with count algorithms,
how should I count the elements in a container satisfying the
condition for large number of elements(ie greater than
difference_type) ?

Kindly clarify.

Thanks
V.Subramanian

In theory, you may be right, but in practice that would be a vector that
contains 4 billion entries, with all the overhead that a vector has per
entry.

Pragmatically speaking, you won't have to worry about overflow problems.

 

[Kai-Uwe Bux]

In theory, you may be right, but in practice that would be a vector that
contains 4 billion entries, with all the overhead that a vector has per
entry.

Actually, in all implementations I know, a vector has no overhead per entry.
It has a small constant overhead since it stores its size and the capacity.

Pragmatically speaking, you won't have to worry about overflow problems.

Agreed.


Best

Kai-Uwe Bux

 

[Kai-Uwe Bux]

You can always create unsigned integer values that have no
corresponding signed value. Similarly, you can always create signed
integer values that have no corresponding unsigned value. That's a
well-known and unsolvable limitation of size_t and ptrdiff_t, and
anything else that tries to talk about sizes and differences.

In the special case of differences between iterators and sizes of
containers, the problem seems solvable: the standard could require (a) the
existence of a signed type sufficiently big to hold all differences between
iterators that can occur on the given implementation and (b) require that
ptrdiff_t and the various difference_type are typedefs for such signed
type.

On a related note: it is even guaranteed that push_back() on a vector cannot
make its size overflow to 0?


Best

Kai-Uwe Bux

 

[kwikius]

What that amounts to is artificially limiting the size of a container to
the non-negative values of a signed type. If you do that, surely someone
will complain that they want their containers to be larger, and don't care
about difference_type.

The real problem is that containers have excessive requirements. arrays and
lookups dont in practise need to be iterable at all, but you can still write
generic algorithms. Remove the iterator paradigm and do for_each (container)
in parallel for example. STL was a major innovator for generic programming
but way way far from perfect.... Too many blooming iterators.

regards
Andy Little

 

[Bo Persson]

In the special case of differences between iterators and sizes of
containers, the problem seems solvable: the standard could require
(a) the existence of a signed type sufficiently big to hold all
differences between iterators that can occur on the given
implementation and (b) require that ptrdiff_t and the various
difference_type are typedefs for such signed type.

Isn't that what happens in practice?

On the most popular 32 bit systems, you would have to try extremely
hard to allocate a vector<char> that uses more than half the available
address space. For any other types, sizes and ptrdiff values are much
smaller.


Bo Persson

 

[kwikius]

Without iterators STL is something entirely different. Container-based
algorithms are nowhere near as flexible as iterator-based ones, because
the sequences that iterators refer to don't need to come from containers.

Most containers arent sequences... except in STL "every container must be a
list". This is a viewpoint imposed by the iterator paradigm.
Custom conforming containers are very complicated to create as need to
provide full custom iterator shebang.

Rather than creating iterators it is just as possible to create views of
arbitrary entities as containers. You only need to create one container view
( as opposed to 2 iterators to create 2 ends of a container view ).

Containers can have simpler requirements... easier to fulfill requirements
and create custom containers

It is also possible to view one container type as another.
increases flexibility for the user and also enables the user to create
custom
containers more easily because each container type has fewer requirements.

But the most important point is it removes the necessity of exposing the
blooming iterators in every algorithm, which makes source code less
expressive :

fun(my_container, your_container , f) ; //overloaded or internally adapted

as against

fun(my_container.begin(), my_container.end(), my_container.begin(),
your_container.end(), f) ; //error?

The loser may be the library implementor of course, but library author to
user is one to many ratio so total code written reduced.
OTOH the library implementor has more scope to optimise because he doesnt
need necessarily to conform internal to the function to the iterator
paradigm.

regards
Andy Little

 

[kwikius]

This is a viewpoint imposed by the need to operate every element in a
container.

Not all containers need to have every element operated on. lookup is an
example. You don't need to iterate a lookup for its main functionality . Now
when you need to iterate over a lookup you can create a listview, and not
just one but many, sorted by key , or key range sorted by value or value
range, arbitrary sort function, and even layer views. You can apply
functionality for the particular purpose when you need it and then discard
it.

If you need to operate on every element in a container, you end up
treating the container as a sequence. Creating one container view versus
two iterators doesn't change that.



Since there is no description here of how this paradigm works, I can't
assess this assertion.

Its about concepts dealing with one single functionality of a type in
expression rather than monolithic concepts covering large amounts of
functionality ( e.g HasPlus preferred to Arithmetic ) think of eg
ForEachableContainer CountIfAbleContainer etc, like the modern move from
"traits blob" to metafunctions.

At the cost of having to pretend that the keyboard, for example, is a
container rather than the source of a sequence of characters.

Oh don't get me started on istream.

Simple console input is a lazy list with customisable terminator... eg for
continuous multiline input, disc istream is array or tuple, database istream
a lookup etc...

regards
Andy Little