H2 hide the type of data a system must use?

[nguillot]

Hello

The subject of this post is not really clear.

Let me try to explain:

I often have this kind of problem:
In a system, a block process some data.
The data are sent by a Sender.
They are asynchronously process by a Processor.
The result of the treatments are sent to a Receiver.

Sender
|
| data
|
\/
Processor ------ results -----> Receiver

The receiver receive results as soon as they are available, and let
say it must save them, and save some informations about the data used
to produce the result, an id for instance.

In a previous development, we simply had:

class Data
{
int id_;

public:
int GetId() const;
}

And:

class Result
{
int id_;

public:
int SetId(int);
}

And so, somewhere, the Processor was doing:

Processor:rocessData(const Data& d)
{
Result res;
res.SetId(d.GetId());

// process data...
}

It was a direct approach, but we finally saw the limitations, mainly:
1) if an int is not enough to tag the data, everything is impacted:
the Data, the sender that produce data, the processor that copy the
part of data needed to tag it, the result and the receiver...
2) a developer can use the id (or tag) to deduce some information used
in the processor. The deduction is true one day, but false latter (to
take a sample, we can deduce from a begin and a end iterator the
element count in a container, by "substracting" them: it may be true
with a container type, and false with another container type).

OK. So the issue is: the processor must do a thing (copy a tag) on
something (the data to the result) without knowing about the data more
than what it needs.
Here it needs to know how to copy, and what to copy, from data to
result.

This is typically the usage, IMHO, of an interface.
So the processor, instead of copying an int get with GetId would get a
ITag pointer. It wouldn't know anything about the concrete class
implementing the tag: it can be an int, or something more complex.
And the processor would call ITag::clone or copy...

I intentionally don't try to code something in this way.

My question is: I think this is a common design issue. Is there a
pattern, a guideline, or some best practices about that?

Something I don't like about the ITag interface is: either the Data
class inherits or aggregates it, nothing force the sender to pass an
object with a tag. Ok, the processor won't compile when trying to
access ITag::clone, but the sender code maybe in another compilation
unit, on another project, and the developer is not aware of the
problem.
We could document, but we know that a programming design is better
than an unread documentation to enforce a practice. To say like Herb
Sutter:
"it would be nice to make compilers reject such code instead of just
making tut-tut noises in standardish legalese."

Thank you if you read until here, and thanks in advance for answer (I
hope).

Finally, again one word: I imagined a way to do that, but it has his
problems:
we could enforce to link a data to a tag by declaring a processor
entry point like that:

Processor:rocessData(CTaggedObject<T, Data> data);

where CTaggedObject is
template <typename Tag, typenam Obj> CTaggedObject
{
Tag tag_;
Obj Obj_;

// constrcutor with tag and object and default values...

const Tag& GetTag() const;
const Obj& GetObject() const;
// and the non const version...
}


So the processor would deal with Data to process data, and it copies
the tag without knowing the Tag type.
The result would be provided as a CTaggedObject<SameTagTypeAsData,
Result>.

What is good:
- The processor doesn't know anything about Tag type (with template
functions it should be possible)
What is not good:
- if the processor is in a separate compilation unit (library) the
compiler doesn't know the Tag type used by other compilation units,
so...
- if the processor is seen by its client (the Sender here) through
a functionnal interface, we cannot have virtual template functions!
nor template virtual function, nor function virtual template nor
function template virtual ;-)

OK, now stop and thanks again for the readers.

 

[nguillot]

I forgot a point for what is good about CTaggedObject: the object
class is not polluted with tag stuff: better encapsulation, easier
code maintenance, evolution...

I read again my post and I saw a lot of errors, no s with with 3rd
person present or plural and so on... pardon my English, it's too late
for a good language

 

[Francesco S. Carta]

Hello

The subject of this post is not really clear.

Let me try to explain:

I often have this kind of problem:
In a system, a block process some data.
The data are sent by a Sender.
They are asynchronously process by a Processor.
The result of the treatments are sent to a Receiver.

Sender
|
| data
|
\/
Processor ------ results -----> Receiver

The receiver receive results as soon as they are available, and let
say it must save them, and save some informations about the data used
to produce the result, an id for instance.

Thank you if you read until here, and thanks in advance for answer (I
hope).

Up to here I was thinking: "templates, templates templates", and finally
you pointed them out.

Finally, again one word: I imagined a way to do that, but it has his
problems:
we could enforce to link a data to a tag by declaring a processor
entry point like that:

Processor:rocessData(CTaggedObject<T, Data> data);

where CTaggedObject is
template<typename Tag, typenam Obj> CTaggedObject
{
Tag tag_;
Obj Obj_;

// constrcutor with tag and object and default values...

const Tag& GetTag() const;
const Obj& GetObject() const;
// and the non const version...
}


So the processor would deal with Data to process data, and it copies
the tag without knowing the Tag type.
The result would be provided as a CTaggedObject<SameTagTypeAsData,
Result>.

What is good:
- The processor doesn't know anything about Tag type (with template
functions it should be possible)

Indeed it is.

What is not good:
- if the processor is in a separate compilation unit (library) the
compiler doesn't know the Tag type used by other compilation units,
so...

Yes, go make a search for "export templates"... there should be at least
one compiler implementing them.

If you are stuck with another compiler, you need to put your templates
in a header library, yes...

- if the processor is seen by its client (the Sender here) through
a functionnal interface, we cannot have virtual template functions!
nor template virtual function, nor function virtual template nor
function template virtual ;-)

I don't see the need for anything virtual, neither I see any difficulty
about passing polymorphic pointers or references... anyway.

All of this stuff made me fiddle with some code, and here below I'll
paste it.

As you will see, the Sender / Processor / Receiver are implemented with
templates and know nothing about the types they will be instantiated
with, they only need those classes to have an "id" member and a "value"
member.

Furthermore, they accept also assigning an id of one type to an id of
another type (assuming a valid conversion operation exists between these
two types) - the DataTemplate is in fact just a wrapper that joins two
unrelated data types to make a single packet with the interface expected
by the triad above - not much different from your CTaggedObject.

Processor is aware of the exact types it's handling via a couple of
typedefs declared into Sender and Receiver.

See if you can find something useful in it:

//-------
#include <iostream>

using namespace std;

// Sender, Processor and Receiver templates

template<class DataType> struct Sender {
typedef DataType data_type;
DataType send() {
DataType data;
cout << "sending id == [" << data.id << "]" << endl;
cout << "sending value == [" << data.value << "]" << endl;
return data;
}
};

template<class ResultType> struct Receiver {
typedef ResultType result_type;
void receive(ResultType result) {
cout << "received id == [" << result.id << "]" << endl;
cout << "received value == [" << result.value << "]" << endl;
}
};

struct Processor {
template<class SenderType, class ReceiverType>
void process(SenderType sender, ReceiverType receiver) {
typename SenderType::data_type data = sender.send();
typename ReceiverType::result_type result;
result.value = data.value;
result.value /= 3;
result.id = data.id;
cout << "transforming id from [" << data.id << "] ";
cout << "to [" << result.id << "] " << endl;
cout << "transforming value from [" << data.value << "] ";
cout << "to [" << result.value << "] " << endl;
receiver.receive(result);
}
};

// test case

template<class Value, class Id> struct DataTemplate {
Value value;
Id id;
DataTemplate() : value(100), id(42) {}
};

int main() {
Sender< DataTemplate<long, unsigned char> > sender;
Receiver< DataTemplate<double, int> > receiver;
Processor processor;
processor.process(sender, receiver);
return 0;
}

/*

output:

sending id == [*]
sending value == [100]
transforming id from [*] to [42]
transforming value from [100] to [33.3333]
received id == [42]
received value == [33.3333]

*/

//-------

 

[tni]

What you are talking about sounds a lot like overengineering (which IMO
is the root of all evil).

Do you really need multiple ID types? If not, don't use interfaces /
templates for the ID.

If your issue with 'int' is the value range, a lot of compilers have
stdint.h with int64_t (if yours doesn't, Boost comes with
boost/cstdint.hpp, there are other portable versions as well).

If you don't want people accessing/manipulating the ID value directly,
stuff the int into a class and provide public methods for the stuff you
want to allow. You can easily change that class later on, as long as
keep the public stuff the same - no need for an abstract interface.

If you are worried about Senders not setting / initializing the ID value
in your tagged object, only provide public constructors / factory
methods that force passing one in (or make the constructor generate one).

\\

> - if the processor is seen by its client (the Sender here) through
> a functionnal interface, we cannot have virtual template functions!
> nor template virtual function, nor function virtual template nor
> function template virtual

Virtual methods and templates work perfectly fine, but you can only
template the whole class, not individual virtual methods.

 

[nguillot]

Thanks for the answers.

What you are talking about sounds a lot like overengineering (which IMO
is the root of all evil).

Well, maybe. However, as this problem is quite frequent, I was asking
for a good practice.

Do you really need multiple ID types? If not, don't use interfaces /
templates for the ID.

Yes I need: the Processor is something common to several applications.
The applications manage their sender and receiver. So the ID type can
be different from one app to another.
Indeed, when another type was needed, the solution was to store the ID
in a generic was: a vector<int>. The ID structure was serialized by
the sender to this list, and deserialized by the receiver.
However, even if this solution is generic, it's not object oriented,
and I would prefer the latter approach.