Interesting question about Templates & RTTI

[sagi.perel]

I have tried to compile the following code on Win & Unix. Doesn't work
on either.
<----- CODE ----->
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <vector>
#include <typeinfo>
using namespace std;

template <class T>
class MyArray
{
private:
vector<T> _data;
public:
void foo(void* mem_buf, int buf_size)
{
if(typeid(T) == typeid(string))
{
char buf[1000]={0};
strncpy(buf, (char*)mem_buf, buf_size);
_data[0] = string(buf);
return;
}

if(typeid(T) == typeid(int))
{
int tmp;
mempcy((void*)&tmp, mem_buf, sizeof(T));
_data[0] = tmp;
return;
}
}
};

int main()
{
MyArray<int> a;
int b =5;
a.foo((void*)&b, sizeof(b));
}

<---- END CODE --->

Win VC8 gives 2 errors:
1) For the line: _data[0] = string(buf); :
error C2440: '=' : cannot convert from
'std::basic_string<_Elem,_Traits,_Ax>' to 'int'
with
[
_Elem=char,
_Traits=std::char_traits<char>,
_Ax=std::allocator<char>
]
No user-defined-conversion operator available that can perform
this conversion, or the operator cannot be called

2) For the line: mempcy((void*)&tmp, mem_buf, sizeof(T));:
error C3861: 'mempcy': identifier not found


Unix g++ 4.1.0 20060304 (Red Hat 4.1.0-3) on i386-redhat-linux gives:
test3.cpp: In member function void MyArray<T>::foo(void*, int):
test3.cpp:28: error: there are no arguments to ×’mempcy that depend on
a template parameter, so a declaration of mempcy must be available
test3.cpp:28: error: (if you use -fpermissive, G++ will accept your
code, but allowing the use of an undeclared name is deprecated)

The problem seems to be that the VC8 compiler is trying to compile the
code with the given T=int, so on the line that says _data[0] =
string(buf);
it is trying to insert a string into an <int> vector. I could dig
that, but it seems annoying that there is no way around that. What I
want to achieve is a template that I can use, where I can parse data
that I get according to its type in runtime.
If there is no other option, I will probably use a base class and
derive from it instead of templates. They just seemed a cleaner
solution. If you have any thoughts- I would love to hear them.
Now, g++ doesn't seem to be bothered by this problem- which is kinda
weird.

The other thing is the problem with memcpy. Anyone can shed a light on
that?
Thanks
Sagi

 

[Alf P. Steinbach]

* (e-mail address removed):

I have tried to compile the following code on Win & Unix. Doesn't work
on either.
<----- CODE ----->
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <vector>
#include <typeinfo>
using namespace std;

template <class T>
class MyArray
{
private:
vector<T> _data;
public:
void foo(void* mem_buf, int buf_size)
{
if(typeid(T) == typeid(string))
{
char buf[1000]={0};
strncpy(buf, (char*)mem_buf, buf_size);
_data[0] = string(buf);
return;
}

if(typeid(T) == typeid(int))
{
int tmp;
mempcy((void*)&tmp, mem_buf, sizeof(T));
_data[0] = tmp;
return;
}
}
};

int main()
{
MyArray<int> a;
int b =5;
a.foo((void*)&b, sizeof(b));
}

The compiler complains because _data[0] is of type 'int'. That in turn
is because you've treated it as being of two different types in the same
piece of code. There are techniques for doing that but those are
compile-time techniques, causing the compiler to not even look at the
code that's not used; for a run-time decision you can't do that since
both codes need to be present to choose one or the other.

How about

template<typename T>
class MyArray
{
private:
std::vector<T> myData;
public:
void setFirstElement( T const& value )
{
myData.at( 0 ) = value;
}

// Whatever, other things.
};

which seems to do all you're attempting with 'foo', only far easier, far
more safe and probably more efficient too (not to mention correct). ;-)

The other thing is the problem with memcpy. Anyone can shed a light on
that?

Be glad that the compiler reported some error.

What book are you using that recommends such evilness as displayed above?

 

[Salt_Peter]

I have tried to compile the following code on Win & Unix. Doesn't work
on either.
<----- CODE ----->
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#include <vector>
#include <typeinfo>
using namespace std;

template <class T>
class MyArray
{
private:
vector<T> _data;

underscores at the start of a var name are reserved.
The above will fail if T is itself a template (hint: std::string) since
<<...>> is parsed as an input stream operator.
MyArray should have a default ctor or a parametized ctor so as to
initialize the member vector.

public:
void foo(void* mem_buf, int buf_size)
{
if(typeid(T) == typeid(string))

Why detect the type? Read below...
Please stop thinking of std::string as a bunch of characters, its a
type, just like int is a type.
In fact its more than just a type, its a dynamic type.

{
char buf[1000]={0};
strncpy(buf, (char*)mem_buf, buf_size);
_data[0] = string(buf);
return;
}

if(typeid(T) == typeid(int))
{
int tmp;
mempcy((void*)&tmp, mem_buf, sizeof(T));
_data[0] = tmp;
return;
}
}
};

int main()
{
MyArray<int> a;
int b =5;
a.foo((void*)&b, sizeof(b));
}

<---- END CODE --->

Let me welcome you to the 21st century:

#include <iostream>
#include <string>
#include <vector>

class Dummy // for observing output
{
public:
Dummy() { std::cout << "Dummy()\n"; }
Dummy(const Dummy& copy) // copy ctor
{
std::cout << "copy Dummy\n";
}
~Dummy() { std::cout << "~Dummy()\n"; }
};

template < class T >
class MyVector
{
std::vector< T > vt;
public:
// def ctor
MyVector() : vt() { }
// parametized ctor
MyVector(const size_t& sz, const T& t = T())
: vt(sz, t) { }
};

int main()
{
MyVector< int > vn(1000000, 99);
MyVector< std::string > vs(10, "string");
MyVector< Dummy > vdum(5);
}

/*
Dummy()
copy Dummy // thats so cool
copy Dummy
copy Dummy
copy Dummy
copy Dummy
~Dummy()
~Dummy()
~Dummy()
~Dummy()
~Dummy()
~Dummy()
*/

It doesn't get any simpler. By the way: std::string already has a
default ctor so:
MyVector< std::string > vs(10);
will do. Ahem - those 10 strings are empty but all dynamic.
The MyVector class would benefit from a copy ctor and assignment
operator, both easy to implement. So now you can take both strncpy and
mempcy and throw them in the garbage.

MyVector< int > vn(1000000, 99);
Just in case you missed it, thats 1 million instant integers all
initialized to 99 on *one* line.

 

[sagi.perel]

Thanks for the reply!
Your suggestion is much more elegant, but it will not solve my problem.
Let me explain.

The code I posted here is just a simplification of what I doing: it's
all part of a system for data messaging between different
architectures.
I am reading a memory buffer from a remote source, and that buffer has
a non constant format.
Along with it, I also get an object holding information which tells me
what the memory buffer represents.
For example, it will tell me that the buffer holds 5 ints, then a
double, then a string sized 45 bytes etc.
I have to "parse" the buffer in real time to get the actual data.

I could make a class for every possible data type, but templates seemed
more convenient, since all those classes would have the exact same
functionality.
So getting a memory buffer is a must for me. I cannot modify the
function to get a T type.

As far as the memcpy- I know it's evil, but I don't know any other fast
way to extract data from a memory chunk into a variable. How would you
extract the value of an int from a (void*) memory chunk?

Hope I made myself clear.
I will be glad if you have any thoughts...
Thanks, Sagi

 

[Nate Barney]

As far as the memcpy- I know it's evil, but I don't know any other fast
way to extract data from a memory chunk into a variable. How would you
extract the value of an int from a (void*) memory chunk?

How about:

int main()
{
int i = 42;
void *p = &i;
int j = *static_cast<int*>(p);
}

 

[sagi.perel]

Thanks Nate!
That is a much safer way.

Do you have any insight about the template problem, or do you figure
that separate classes for different data types would be the best
solution?

 

[Emmanuel Deloget]

(e-mail address removed) a écrit :

Thanks Nate!
That is a much safer way.

Do you have any insight about the template problem, or do you figure
that separate classes for different data types would be the best
solution?

It's not safer than a memcpy version, although at least it's more
elegant.

For your template problem, I think you have to make a design decision
here: if you need to know the real type of T when you process the data,
then something is wrong - as this is a clear breach of the Open/Closed
principle (see wikipedia or c2.com for more info). Templates should
allow you to create code that is generic. Reintroducing type name into
this don't make much sense IMHO.

I guess that the data you are processing comes from some sort of file,
maybe a socket (that would explain why you need to use a raw buffer +
its size ; if it's not the case, please give us more information about
what you want to achieve, as there may be a better solution). A
classical solution for this problem is to use a helper template
function that can be specialized. For example:

#include <algorithm> // for std::copy

template <class T>
void copy_from_raw(T& dest, void *buf, std::size_t size)
{
// unspecialized, raw copy:
char *temp = reinterpret_cast<char*>(&buf);
int min_size = std::min(sizeof(T), size)
std::copy(temp, temp+min_size, reinterpret_cast<char*>(&temp));
}

template <>
void copy_from_raw<int>(int& dest, void *buf, std::size_t size)
{
dest = *(reinterpret_cast<int*>(buf));
}

template <>
void copy_from_raw<std::string>(string& dest, void *buf, std::size_t
size)
{
dest.resize(size);
char *temp = reinterpret_cast<char*>(&buf);
std::copy(temp, temp + size, dest.begin());
}

(I used reinterpret_cast instead of static_cast because I want to make
clear that the void* source is really unrelated to the destination
type).

In the code, you use:

template <class T>
class my_array
{
std::vector<T> v;
public:
void foo(void *buf, std::size_t size)
{
T dest;
// if T is int, it will call copy_from_raw<int>(); if T is a string
it will call
// copy_from_raw<std::string>(); otherwise it will call the generic
version
// of copy_from_raw<T>()
copy_from_raw(dest, buf, size);
v.push_back(dest);
}
};

The reason why you use an external function is that you can easily
specialize it, which keep your template code generic. There are some
example of this technique in the standard library itself.

HTH,

-- Emmanuel Deloget, Artware

 

[sagi.perel]

Dear Emmanuel Deloget
Thank you for your reply. Your suggestion did solve my problem!
The external functions do enable the template code to remain general,
and this is exactly what I needed. Too bad the C++ books I have do not
mention such a design. They do mention typeid as a solution which is a
lot uglier.

As par your questions:
the data is coming from sockets and pipes, some of it directly from
hardware drivers that I do not have control over; and I would hate to
make a wrapper for them to format the data for me.
The module that I am building interfaces with various other processes
and collects data from them. The data format changes in runtime, but
the header of each message that I get specifies the format.

You said that cast is just as safe as memcpy. Is that the case? memcpy
can cause you to overrun a buffer.
Will cast do the same?
I can imagine that interpret_cast would, if you are forcing a cast to
something that is bigger than the real data.

Thanks again!

 

[Emmanuel Deloget]

(e-mail address removed) a écrit :

Dear Emmanuel Deloget
Thank you for your reply. Your suggestion did solve my problem!
The external functions do enable the template code to remain general,
and this is exactly what I needed. Too bad the C++ books I have do not
mention such a design. They do mention typeid as a solution which is a
lot uglier.

Most books that have been written in the 2000-2003 era are very
object-oriented. While the procedural+OO mix is not new, it has (IMHO)
not been seen as a very valid programming paradigm until recently
(despite the STL architecture). This is changing rapidly, as this mix
enables you to express not only things (objects) but also actions
(functions). However, this is quite out of subject here

<snip>

You said that cast is just as safe as memcpy. Is that the case? memcpy
can cause you to overrun a buffer.
Will cast do the same?
I can imagine that interpret_cast would, if you are forcing a cast to
something that is bigger than the real data.

Whenever you use a cast or use memcpy (which will either requires an
explicit cast or do an implicit cast), you quit the wonderful realm of
type safety, and the ugly witch of the dark forest of the bugs will now
be able to catch you and boil you. As a consequence, it's far better to
avoid any solution that use void* or pointer type cast when possible.
Now, you're also true that memcpy also allow you to override memory
(but then a pointer cast allow you to do very nasty things as well).

Thanks again!

You're welcome

-- Emmanuel Deloget, Artware