template and dynamic decisions

[Jocke P]

Hi,

I'm trying to define a file reader class which is endian-aware.
My problem is that descendants should select at creation time which
endianness to use, and the base class has all file reading methods,
which I think should be declared as templates:

template<class T> size_t FileReader::read_n(T* tptr, size_t ncount);

But now I'm confused how to set up the byte swapping. Whether to swap
should be configured by a descendant at construction time, depending on
file byte-order.
Here's the intended implementation of the read_n method:

template<class T> filesize_t FileReader::read_n(T* tptr, size_t ncount)
{
size_t n = readRaw(tptr, ncount * sizeof(T));
// Swapper functional should be defined as LITTLEend or BIGend
// depending on file byte-order.
// These in turn are #defined as ByteSwap or DoNothing
// depending on native byte-order.
std::transform(tptr, tptr + ncount, tptr, Swapper);
return n;
}

How could descendants select the swapper functional?
The read() method cannot be virtual since virtual methods cannot be templates.
For the same reason the transform call couldn't be replaced by a
virtual method.
I could of course do something through-and-through dynamic, like adding
an enum or bool member set by descendant, and switch on it in the read methods:

template<class T> filesize_t FileReader::read_n(T* buffer, size_t ncount)
{
filesize_t nbytes = readRawN(buffer, ncount * sizeof(T));
if (_endian == MyLittleEnum)
std::transform(buffer, buffer + ncount, buffer, LITTLEend);
else // (_endian == MyBigEnum)
std::transform(buffer, buffer + ncount, buffer, BIGend);
return nbytes;
}


But this seems silly since endianness is known at descendant creation time.
Isn't there some better alternative?
I wouldn't like to make template of the file reader base class,
since most of its code is not endian dependent, and on my favourite
Defective Compiler I think it would prevent descendants to have template
methods (since partial templ specialization is not accepted).

(PS. I seem to run into very similar problem with templates all the time,
so it would be grand with some entries on template syntax in the FAQ Lite,
though obviously I couldn't offer to write them...)

Thanx for any input,

Jocke

 

[John Harrison]

Hi,

I'm trying to define a file reader class which is endian-aware.
My problem is that descendants should select at creation time which
endianness to use, and the base class has all file reading methods,
which I think should be declared as templates:

template<class T> size_t FileReader::read_n(T* tptr, size_t ncount);

But now I'm confused how to set up the byte swapping. Whether to swap
should be configured by a descendant at construction time, depending on
file byte-order.
Here's the intended implementation of the read_n method:

template<class T> filesize_t FileReader::read_n(T* tptr, size_t ncount)
{
size_t n = readRaw(tptr, ncount * sizeof(T));
// Swapper functional should be defined as LITTLEend or BIGend
// depending on file byte-order.
// These in turn are #defined as ByteSwap or DoNothing
// depending on native byte-order.
std::transform(tptr, tptr + ncount, tptr, Swapper);
return n;
}

How could descendants select the swapper functional?
The read() method cannot be virtual since virtual methods cannot be templates.
For the same reason the transform call couldn't be replaced by a
virtual method.
I could of course do something through-and-through dynamic, like adding
an enum or bool member set by descendant, and switch on it in the read methods:

template<class T> filesize_t FileReader::read_n(T* buffer, size_t ncount)
{
filesize_t nbytes = readRawN(buffer, ncount * sizeof(T));
if (_endian == MyLittleEnum)
std::transform(buffer, buffer + ncount, buffer, LITTLEend);
else // (_endian == MyBigEnum)
std::transform(buffer, buffer + ncount, buffer, BIGend);
return nbytes;
}


But this seems silly since endianness is known at descendant creation time.
Isn't there some better alternative?
I wouldn't like to make template of the file reader base class,
since most of its code is not endian dependent, and on my favourite
Defective Compiler I think it would prevent descendants to have template
methods (since partial templ specialization is not accepted).

(PS. I seem to run into very similar problem with templates all the time,
so it would be grand with some entries on template syntax in the FAQ Lite,
though obviously I couldn't offer to write them...)

Thanx for any input,

Jocke

What about making the swapper function another template parameter?

#include <algorithm>

class FileReader
{
public:
template<class T, T (*Swapper)(const T&)>
size_t read_n(T* tptr, size_t ncount)
{
std::transform(tptr, tptr+ncount, tptr, Swapper);
return ncount;
}
};

template <class T> T BIGend(const T&);
template <class T> T LITTLEend(const T&);

int main()
{
FileReader r;
int a[100];
r.read_n<int, BIGend<int> >(a, 100);
}

john

 

[Gianni Mariani]

Hi,

I'm trying to define a file reader class which is endian-aware.
My problem is that descendants should select at creation time which
endianness to use, and the base class has all file reading methods,
which I think should be declared as templates:

An alternative is to allways write the file in the same endianness.

That way you can determine at compile time, which endianness to use.
I have written a class (NetworkOrder) and posted a couple of times.
It's available from google:

http://groups.google.com/groups?hl=...8&edition=us&[email protected]

This fancy method determines the endianness of the machine. With full
optimization on, it's code is elminated and is equivalent to a constant.

static inline bool IsBigEndianbool()
{
const unsigned x = 1;
return ! ( * ( const char * )( & x ) );
}

Now you can write a single method that determines wether to swap or not
with an if ().

template<class T> size_t FileReader::read_n(T* tptr, size_t ncount);

But now I'm confused how to set up the byte swapping. Whether to swap
should be configured by a descendant at construction time, depending on
file byte-order.
Here's the intended implementation of the read_n method:

template<class T> filesize_t FileReader::read_n(T* tptr, size_t ncount)
{
size_t n = readRaw(tptr, ncount * sizeof(T));
// Swapper functional should be defined as LITTLEend or BIGend
// depending on file byte-order.
// These in turn are #defined as ByteSwap or DoNothing
// depending on native byte-order.
std::transform(tptr, tptr + ncount, tptr, Swapper);
return n;
}

How could descendants select the swapper functional?
The read() method cannot be virtual since virtual methods cannot be
templates.
For the same reason the transform call couldn't be replaced by a
virtual method.
I could of course do something through-and-through dynamic, like adding
an enum or bool member set by descendant, and switch on it in the read
methods:

template<class T> filesize_t FileReader::read_n(T* buffer, size_t ncount)
{
filesize_t nbytes = readRawN(buffer, ncount * sizeof(T));
if (_endian == MyLittleEnum)

if ((_endian != MyBigEndianEnum) && IsBigEndianbool())

std::transform(buffer, buffer + ncount, buffer, SwapMethod);

return nbytes;
}


But this seems silly since endianness is known at descendant creation time.
Isn't there some better alternative?

Allways write the file big endian. or Allways write the file little
endian.

Or you can allways create two classes that read, one that swaps and one
that does not with a virtual read method.

 

[Kevin Goodsell]

An alternative is to allways write the file in the same endianness.

Good idea.

That way you can determine at compile time, which endianness to use.

Or you could write code that is independent of endianness. Writing code
to handle every possible byte order seems excessive, problematic, and
unnecessary.

I have written a class (NetworkOrder) and posted a couple of times. It's
available from google:

http://groups.google.com/groups?hl=...8&edition=us&[email protected]


This fancy method determines the endianness of the machine. With full
optimization on, it's code is elminated and is equivalent to a constant.

static inline bool IsBigEndianbool()
{
const unsigned x = 1;
return ! ( * ( const char * )( & x ) );
}

But... this tells you very little, really. In fact, it can return true
for something that is NOT big-endian.

I just think that this is a fundamentally bad way of doing file I/O. I
would use something like this instead:

int BytesToInt(unsigned char bytes[])
{
int result = 0;
for (int i=0; i<sizeof(int); ++i)
{
result = (result << CHAR_BIT) | bytes;
}
}

Of course this could be made much more general and safe with a few
modifications. For example: it could be made a template to handle
different types, it could use a vector instead of an array to pass the
bytes in, it could indicate a number of bytes to convert and throw an
exception if the number of bytes to convert exceeds the number of bytes
in the destination type, etc.

-Kevin

 

[Gianni Mariani]

Good idea.



Or you could write code that is independent of endianness. Writing code
to handle every possible byte order seems excessive, problematic, and
unnecessary.



But... this tells you very little, really. In fact, it can return true
for something that is NOT big-endian.

OK - you got me - without going to the PDP-11. How ?

I just think that this is a fundamentally bad way of doing file I/O. I
would use something like this instead:

int BytesToInt(unsigned char bytes[])
{
int result = 0;
for (int i=0; i<sizeof(int); ++i)
{
result = (result << CHAR_BIT) | bytes;
}
}

Of course this could be made much more general and safe with a few
modifications. For example: it could be made a template to handle
different types, it could use a vector instead of an array to pass the
bytes in, it could indicate a number of bytes to convert and throw an
exception if the number of bytes to convert exceeds the number of bytes
in the destination type, etc.

Did you check the link in the previous post ?

 

[Jocke P]

This fancy method determines the endianness of the machine. With full
optimization on, it's code is elminated and is equivalent to a constant.

static inline bool IsBigEndianbool()
{
const unsigned x = 1;
return ! ( * ( const char * )( & x ) );
}

Thanks for mentioning the trick. I did try to google for a
#define-free detection some time ago, but didn't find it.
After you mention, I could find it in a few places.

Anyway, the method above requires an addressable item,
thus I couldn't get it accepted as a template parameter.
So I tried the following simple variation:

struct MachineEndian
{
enum { is_big = ('B' == (char)(int)'Big ') };
};


- and it "works" (ie is 0) on my little-endian machine.
But...
Is it legal? (I mean the four-char '1234' construct)
Is it portable?

Oh, first maybe I should ask, does it work as intended
on a big-endian...?


Cheers,

jp

 

[Gianni Mariani]

Thanks for mentioning the trick. I did try to google for a
#define-free detection some time ago, but didn't find it.
After you mention, I could find it in a few places.

Anyway, the method above requires an addressable item,
thus I couldn't get it accepted as a template parameter.
So I tried the following simple variation:

struct MachineEndian
{
enum { is_big = ('B' == (char)(int)'Big ') };
};


- and it "works" (ie is 0) on my little-endian machine.
But...
Is it legal? (I mean the four-char '1234' construct)
Is it portable?

Oh, first maybe I should ask, does it work as intended
on a big-endian...?

On gcc 3.3.1 I get
warning: multi-character character constant

But 2 character "char" constants I though were part of the C standard
which would make them highly likely part of the C++ standard.

I'm pretty sure this does not do the right thing in a cross-comple.
What I think you're testing here is the endianness of the compiler and
not really the endianness of the target.

I would vote for not portable, but I'm not sure.

It would be cool if it was guarenteed (and gcc would not chirp warnings).

 

[Ron Natalie]

On gcc 3.3.1 I get
warning: multi-character character constant

But 2 character "char" constants I though were part of the C standard
which would make them highly likely part of the C++ standard.

When you have more than one character in the character constant, the
interpretation is implementation-defined. G++ is warning you that you
many not really want to do that (but it does accept it).

It's obviously not a good test because the implementation definedness doesn't
necessarily depend on the byte order.

 

[Ron Natalie]

I say escalate this issue to the C++ gods and get it to be made well
defined in the next C++ standard.

Why, what possible use is it?

Let's say type int is 32 bits and char is 8 bits. If we define
a character constant 'big'? How would this be packed?
Would it be left justified? right justified? MSB first, LSB first?

It is addressed by the stanard. If you want a char, use a char. If
you want some other type, don't initialize it with a char literal.

 

[Gianni Mariani]

Why, what possible use is it?

Let's say type int is 32 bits and char is 8 bits. If we define
a character constant 'big'? How would this be packed?
Would it be left justified? right justified? MSB first, LSB first?

It is addressed by the stanard. If you want a char, use a char. If
you want some other type, don't initialize it with a char literal.

If the standard allows 'ABCD' as a character literal, then it should
define what it does in an unambiguius way. If it does not then it's
silly to allow for it in the standard.

Anyhow - I just remembered I have a g++ - solaris cross compiler loaded
on my machine (which is big endian) and the it actually does the
(TM)right thing.

I think we need to pull the standard to see. If the GCC folks went to
the trouble to make this work correctly, I think there has to be a good
reason.

G

 

[Gianni Mariani]

Ron Natalie wrote:
....


Anyhow - I just remembered I have a g++ - solaris cross compiler loaded
on my machine (which is big endian) and the it actually does the
(TM)right thing.

The statement above is blatently wrong and even incorrect.

A g++ cross compiler does in fact NOT do the (TM)right thing.

 

[Samuel Barber]

struct MachineEndian
{
enum { is_big = ('B' == (char)(int)'Big ') };
};

- and it "works" (ie is 0) on my little-endian machine.

Why do you call that working? If we assume for the sake of argument
(NOT TRUE!) that '1234' depends on endianness, then it should work
like this:

little endian: '1234' = 0x34333231
big endian: '1234' = 0x31323334

So your test failed.

If that isn't clear, consider:

#include <stdio.h>

int main(void)
{
char a[5]="1234";

int i = *(int*)a;

int j = '1234';

// does i == j ?

printf("i=0x%x, j=0x%x", i, j);

return 0;
}

By the way, Intel assemblers use the "big endian" definition of
'1234'.

Sam