Getting some code into container classes

[woodbrian77]

I'd like to see stream constructors like this added to
container classes.

#ifdef CMW_DREAMER
#include <ReceiveBuffer.hh>
#endif


template < ...>
class container {
...

#ifdef CMW_DREAMER
template <class R>
explicit container :cmw::ReceiveBuffer<R>& buf)
{
int32_t count=buf.template Give<uint32_t>();
for(; count>0; --count)
emplace_back(value_type{buf});
}
#endif

};

If the value_type was numeric, one line would have
to change:

emplace_back(buf.template Give<value_type>());

How to integrate those two things?

Having stream constructors would help me get
away from outputting code like this:

template <class R>
void Give :cmw::ReceiveBuffer<R>& buf
,std::vector<std::deque<std::string> >& az1){
int32_t count[2];
count[0]=buf.template Give<uint32_t>();
az1.reserve(az1.size()+count[0]);
for(;count[0]>0;--count[0]){
std::deque<std::string> rep2;
count[1]=buf.template Give<uint32_t>();
for(;count[1]>0;--count[1]){
rep2.emplace_back(buf.GiveString());
}
az1.emplace_back:std::move(rep2));
}
}

And toward code like this:

template <class R>
void Give :cmw::ReceiveBuffer<R>& buf
,std::vector<std::deque<std::string> >& az1){
std::vector<std::deque<std::string> >{buf}.swap(az1);
}

I realize the semantics are different between those
two, but am not sure it matters.

Would Leigh humor me by adding something like this
to segmented_array? Tia.

Brian
Ebenezer Enterprises
http://webEbenezer.net

 

[woodbrian77]

Not a chance mate; your approach is wrong: modifying containers is not
the answer.

Previously I posted this:

https://groups.google.com/forum/#!topic/comp.lang.c++/Yfhdw5TSwkA


Brian
Ebenezer Enterprises
webEbenezer.net

 

[Öö Tiib]

I'd like to see stream constructors like this added to
container classes.

What streams do "Give"? Streams have iterators:

typedef std::vector<std::string> Texts;
typedef std::istream_iterator<std::string> StreamTextIn;

Texts texts(StreamTextIn(std::cin), StreamTextIn());

 

[woodbrian77]

What streams do "Give"? Streams have iterators:

typedef std::vector<std::string> Texts;
typedef std::istream_iterator<std::string> StreamTextIn;

Texts texts(StreamTextIn(std::cin), StreamTextIn());

It could be called a deserialization constructor.
Did you look at the stackoverflow thread?

Brian
Ebenezer Enterprises
http://webEbenezer.net

 

[Öö Tiib]

It could be called a deserialization constructor.

Yes. I would go with iterators and with 'operator<<' and
'operator>>'. All standard containers have already
constructors that take iterators and the operators are
idiomatic in C++ (regardless what C guru Jakob Navia
says).

The only case when you really *must* avoid two-phase
construction is with objects that lack "empty" or "missing"
state. None of standard containers is such but they still
have constructors that accept input iterators.

Did you look at the stackoverflow thread?

Yes. However I am not sure what there is to look? Most
popular answer makes that:

Foo FooDeserialiser(Data data);

I would just use idiomatic things like ...:

Data& operator>>(Data& in, Foo& p) { /* ... */ }

.... and/or ...

template <class InputIterator>
Data:ata(InputIterator first, InputIterator last, OptionalOtherStuff o ) { /* ... */ }

... instead.

 

[Öö Tiib]

template <class InputIterator>
Data:ata(InputIterator first, InputIterator last, OptionalOtherStuff o ) { /* ... */ }

In light of OMGtechy's answer in
http://stackoverflow.com/questions/...ine-operator-for-my-specialization-of-stdpair
that would make more sense:

template <class InputIterator>
Foo::Foo(InputIterator first, InputIterator last, OptionalOtherStuff o ) { /* ... */ }

I'm not sure why so lot of people use nonsense names in their examples
like 'B', 'Data' and 'Foo'. Gets me confused so when I try to suggest
alternative I tend to mix up what was meant to be what in original.

 

[Luca Risolia]

#ifdef CMW_DREAMER
template <class R>
explicit container :cmw::ReceiveBuffer<R>& buf)
{
int32_t count=buf.template Give<uint32_t>();
for(; count>0; --count)
emplace_back(value_type{buf});
}
#endif

If the value_type was numeric, one line would have
to change:
emplace_back(buf.template Give<value_type>());

How to integrate those two things?

template <class T>
using is_numeric = std::is_arithmetic<T>;

template<class Buf, class T = value_type>
typename std::enable_if<is_numeric<T>::value, void>::type
_emplace_back(Buf& buf) {
emplace_back(buf.template Give<value_type>());
}

template<class Buf, class T = value_type>
typename std::enable_if<!is_numeric<T>::value, void>::type
_emplace_back(Buf& buf) {
emplace_back(value_type{buf});
}

template <class R>
explicit container :cmw::ReceiveBuffer<R>& buf) {
int32_t count = buf.template Give<uint32_t>();
for(; count>0; --count)
_emplace_back(buf);
}

 

[woodbrian77]

Yes. I would go with iterators and with 'operator<<' and
'operator>>'. All standard containers have already
constructors that take iterators

I'm not sure how I could produce the second
iterator. I could probably produce the first,
but not the last. If it's a container of variable
length objects, I don't know where the last one
starts/ends.

Brian
Ebenezer Enterprises - In G-d we trust.
http://webEbenezer.net

 

[Öö Tiib]

I'm not sure how I could produce the second
iterator. I could probably produce the first,
but not the last. If it's a container of variable
length objects, I don't know where the last one
starts/ends.

Note that receiving end rarely knows end of stream and so
there are input iterators. Read up on iterators, standard
library uses them rather heavily.

Basically, input iterators of non-dereferencable state
compare equal with each other and not equal with iterators
of dereferencablestate. You can provide an iterator in
non-dereferencable state as second iterator of range.
I already gave example code.

typedef std::vector<std::string> Texts;
typedef std::istream_iterator<std::string> StreamTextIn;

Texts texts(StreamTextIn(std::cin), StreamTextIn());

Doesn't it work? Didn't try but it was not meant as fiction.

 

[woodbrian77]

Note that receiving end rarely knows end of stream and so
there are input iterators. Read up on iterators, standard
library uses them rather heavily.

I don't think that would scale well.
A container's value_type can be another container,
a user defined type or a primitive type. If containers
have a deserialization constructor, I could write

std::vector<std::deque<std::string> >{buf}

I guess you are saying something like:

std::vector<std::deque<std::string> > (MyIterator(buf), MyIterator())

That works at the top level, but it doesn't work at
the next level for the deques. (I know deque has a
ctor that takes iterators, but there's no place to
supply the iterators.)

Part of the difficulty has to do with the two parameters.
If containers took a "range" object it would be easier.
I remember some discussion of that subject a few years
ago, but I don't know where that stands.

I'm not sure ranges are the answer though either.
I think they'd force me to deal with items one by one.
If an array<double, 1000000> is expected, there's no
way to use an optimization?


Brian
Ebenezer Enterprises - In G-d we trust.
http://webEbenezer.net

 

[Öö Tiib]

I don't think that would scale well.
A container's value_type can be another container,
a user defined type or a primitive type. If containers
have a deserialization constructor, I could write

std::vector<std::deque<std::string> >{buf}

Yes, you have already said that.

I guess you are saying something like:

std::vector<std::deque<std::string> > (MyIterator(buf), MyIterator())

I am writing (third time perhaps?):

typedef std::deque<std::string> Stuff;
typedef BufIterator<Stuff> StuffIn;

That works at the top level, but it doesn't work at
the next level for the deques. (I know deque has a
ctor that takes iterators, but there's no place to
supply the iterators.)

Why can't 'BufIterator<std::deque<std::string>>'

Part of the difficulty has to do with the two parameters.
If containers took a "range" object it would be easier.
I remember some discussion of that subject a few years
ago, but I don't know where that stands.

What difficulty? Your description of issues is a bit
like FUD.

I'm not sure ranges are the answer though either.
I think they'd force me to deal with items one by one.
If an array<double, 1000000> is expected, there's no
way to use an optimization?

What optimization? How can a constructor of standard
library container access your "buf" more optimally than
special "buf_iterator" made by you?

 

[woodbrian77]

I am writing (third time perhaps?):

typedef std::deque<std::string> Stuff;
typedef BufIterator<Stuff> StuffIn;
std::vector<Stuff> Thing(StuffIn(buf), StuffIn());

Thanks for putting that into my terms. It helps a
little. In the original post I called reserve() on
the vector after getting the number of elements.
That looks like an optimization that's lost here.

Why can't 'BufIterator<std::deque<std::string>>'


What difficulty? Your description of issues is a bit
like FUD.


What optimization? How can a constructor of standard
library container access your "buf" more optimally than
special "buf_iterator" made by you?

Using copy/memcpy on a block of data. Maybe the
optimization is possible with what you are saying,
but I haven't understood how it would be done.


Brian
Ebenezer Enterprises - Heavenly code.
http://webEbenezer.net

 

[Öö Tiib]

Thanks for putting that into my terms. It helps a
little. In the original post I called reserve() on
the vector after getting the number of elements.
That looks like an optimization that's lost here.

You either have cheap way to predict the end of
sequence and distance before-hand or not. If you
have it then use it, if you don't have it then you
can't use it. If you want some static 'maybe' to work
here then you use templates; if you want dynamic
'maybe' to work then you use dynamic polymorphism. You
can maybe want to read up on SCARY iterators and why
those are good.

Also you can use the second form that I also wrote
about already:

template<typename Stuff,typename Alloc>
Buf& operator>>(Buf& in, std::vector<Stuff,Alloc>& p)
{ /* ... */ }

Usage of 'reserve' can anyway happen only in 3-step
initialization:

Something something;
// next two steps can be done by 'operator>>'
something.reserve( predictSize() );
fillItSomehow( something );

Or why can't 'BufIterator<std::deque<std::string>>' use

Using copy/memcpy on a block of data. Maybe the
optimization is possible with what you are saying,
but I haven't understood how it would be done.

Uhh ... you got me started.

Code of standard containers won't be required to
contain 'memcpy' *ever*. You either have to write it
explicitly somewhere or learn what loops particular
compiler optimizes into 'memcpy'. 'std::array' won't
*ever* have any written constructors; all will be
compiler-generated. 'array' is required to be
aggregate. If 'array's contents come from dynamic
source then you have to fill it with your function
(say 'operator>>'). Function is yours and so the
possible 'memcpy' is yours.

Dynamic container of standard library may use
'memcpy' (again written by you) during construction
and other usages. It is because elements of such
containers are usually required to be copied, moved
or otherwise constructed by calling
'allocator_traits<allocator_type>::construct'
and allocator may be any class given by you and its
traits may be specialized by you and so there are
lot of places for 'memcpy' written by you.

Currently most powerful optimization with dynamic
containers is move (not memcpy). Move means that if
the 'allocator' is same then the ownership of internals
is transferred, otherwise elements are move-constructed
one-by-one. Needlessly copying too lot of crap around
in huge and slow memory-space of modern hardware
(regardless if with that error-prone 'memcpy' or
otherwise) is typical cause of performance issues.

For my taste there are already too lot of alternative
ways to inject ones will into behavior of standard
containers. It is lot to learn so people don't. It is
impossible to micromanage and improve work of standard
library without knowing what it does. However people
are humans with their FUD about things they sense their
extent of ignorance about.

Strange result is that people ask for some yet another
"magic" way. It is even *more* impossible that standard
library will extend itself with something that knows
what to do with yet another strange "buf" that "Give"s
and uses opportunity to "memcpy" it.

Learn what standard library does. Both 'vector' and
'deque' have already 6 constructors one of what is
template. Then if some other container you want to
extend (say Leigh wrote it or you took it from boost)
is not so over-abundantly flexible then you can
contact the authors with improvement requests that
make sense.