C++ IO (or equivalency of perl's pack/unpack?)

[Stacy Mader]

Greetings all,

I have a VMS binary file with weather data. The record
on each line is 1xint(8) 12xint(2)

Using the perl unpack function, I can decode the
binary file like this:

<snip>

my $template="x8v12"; #v = short in "VAX" (little-endian) order.
my $recordsize=length(pack($template,()));

my($record,$string);

while(read(IN,$record,$recordsize)) {
my(@fields) = unpack($template,substr($record,2,32));
... process @fields here
}
}

<snip>


Does anyone know of a way to do this in C++ ? I have yet
to find a library which can help me...

Regards and thanks in advance,

Stacy.

 

[Ekkehard Morgenstern]

Does anyone know of a way to do this in C++ ? I have yet
to find a library which can help me...

There's no standard way of doing this that I knew of.

However, you could write it yourself, given that you know the byte order of
the integers in the file.

on each line is 1xint(8) 12xint(2)

I figure the first field is a 64 bit integer, right? Followed by 12 16-bit
integers?

Assuming that your C++ compiler has a 64-bit integer type called "_int64"
and a 16-bit integer type called "short", and an 8-bit integer type called
"char", you could do something like this:

typedef unsigned _int64 field8_t;
typedef unsigned short field2_t;
typedef unsigned char byte_t;

#pragma pack(2) // this is for structure packing alignment on
2-byte-boundary
struct record_t {
field8_t f8;
field2_t f2;
};
#pragma pack() // returns to default structure packing.

bool read_record( record_t& rec ) {
byte_t b[8];
cin >> b[0] >> b[1] >> b[2] >> b[3] >> b[4] >> b[5] >> b[6] >> b[7];
// the following composes the bytes read into a 64 bit integer field;
change the order of the bytes to the byte order you need (I assumed MSB
stored first):
rec.f8 = ( (field8_t) b[0] << 56 ) | ( (field8_t) b[1] << 48 ) | (
(field8_t) b[2] << 40 ) | ( (field8_t) b[3] << 32 ) |
( (field8_t) b[4] << 24 ) | ( (field8_t) b[5] << 16 ) | (
(field8_t) b[6] << 8 ) | ( (field8_t) b[7] );
// same goes for the 2-byte fields
for ( int i=0; i<12; ++i ) {
cin >> b[0] >> b[1];
// change the byte order to meet your requirements (I assumed MSB
stored first)
rec.f2 = ( (field2_t) b[0] << 8 ) | ( (field2_t) b[1] );
}
return true;
}

I hope this helps.

 

[Lally Singh]

Greetings all,

I have a VMS binary file with weather data. The record
on each line is 1xint(8) 12xint(2)

Using the perl unpack function, I can decode the
binary file like this:

<snip>

my $template="x8v12"; #v = short in "VAX" (little-endian) order.
my $recordsize=length(pack($template,()));

my($record,$string);

while(read(IN,$record,$recordsize)) {
my(@fields) = unpack($template,substr($record,2,32));
... process @fields here
}
}

<snip>


Does anyone know of a way to do this in C++ ? I have yet
to find a library which can help me...

Regards and thanks in advance,

Easiest way's probably to define a struct with your contents in the
right order.
Watch out for padding between structure members.
I'm not exactly sure what you mean by "1xint(8) 12xint(2)", but try
this:
struct rec {
long long eight_byte_int;
short two_byte_ints[12];
}
Then some code like this:

int fd = open ("filename", O_RDONLY);
rec r;
while (read(fd, &r, sizeof(r)))
do_something_with_data(rec);
close(fd);

htonl() and ntohl() may help with endianness, but they're only good
for 32bits. Writing your own endianness converters isn't too bad.

 

[Alex Lyman]

Greetings all,

I have a VMS binary file with weather data. The record
on each line is 1xint(8) 12xint(2)

Using the perl unpack function, I can decode the
binary file like this:

<snip>

my $template="x8v12"; #v = short in "VAX" (little-endian) order.
my $recordsize=length(pack($template,()));

my($record,$string);

while(read(IN,$record,$recordsize)) {
my(@fields) = unpack($template,substr($record,2,32));
... process @fields here
}
}

<snip>


Does anyone know of a way to do this in C++ ? I have yet
to find a library which can help me...

Perl's pack() is just a fancy way to grab the data from a stored struct {}:

/* Begin Code */
FILE* filehandle;
struct weatherData { // Might want to have your struct definition global
instead, but this suits my purposes.
// Name these better:
short first[4]; // The first 8 bytes (which you, above, I don't believe
you stored -- due to the 'x' pack-type -- but we have to in C).
short last; // Above you used the 'v' pack-type to store it.
} Record; // this gives us the memory to store a single record.
/* Open the file and what-not here */
read(filehandle, &record, sizeof(weatherData)); // Gets a single record from
the current file position, and stores it in Record.
/* End Code */

You'll note LIBC's read() is pretty much synonymous with Perl's read(),
since Perl calls it directly (with some wrapper to translate between Perl's
[GLOB, SCALAR, SCALAR] types and LIBC's [FILE*, void*, size_t] types).

If your target system is not also little-endian (as your data file is),
you'll need something that switches endianness for the read-in data, too --
Perl did this for you because you used the 'v' pack-type -- C isn't so
forgiving. You might be able to find this in your system's LIBC, but I
don't know what it'd be called.

- Alex

 

[Ekkehard Morgenstern]

I have to correct a bug here that I overlooked:

#pragma pack(2) // this is for structure packing alignment on
2-byte-boundary
struct record_t {
field8_t f8;
field2_t f2[12]; // <- of course this should be an array
};
#pragma pack() // returns to default structure packing.

// same goes for the 2-byte fields
for ( int i=0; i<12; ++i ) {
cin >> b[0] >> b[1];
// change the byte order to meet your requirements (I assumed MSB
stored first)
rec.f2 = ( (field2_t) b[0] << 8 ) | ( (field2_t) b[1] ); //

<-- here, the array index was missing