problems with unpack() in Linux, Perl v5.8.0

[Avalon1178]

Hi,

I'm trying to write a PERL program that parses a binary file which
essentially contains the following format:

<float><int><int><int><int><int>

This "record" is repeated throughout the binary file. I'm using
unpack() to extract that information, however, the results I'm seeing
are getting corrupted in PERL v5.8.0 (Linux). I ran the same PERL code
on v5.6.1 Linux (I downloaded it from ActiveState, but didn't "install"
it on my linux system...just cd'ed to the location where the perl
executable lives from this package) and everything works fine. Has
this been observed by anyone else? Is this a known bug in v5.8.0?

Here's the code snippet that I wrote:

$template = "f I5";
$recordsize = length(pack($template,( )));
open(IN, "binFile.bin") or die "Cannot open file";
while (read(IN, $record, $recordsize))
{
($f, $i_1, $i_2, $i_3, $i_4, $i_5) = unpack($template, $record);
print "f=$f, i_1=$i_1, i_2=$i_2, i_3=$i_3, i_4=$i_4, i_5=$i_5\n";
}

If anyone can help I will greatly appreciate it. Thanks.

AV

 

[Avalon1178]

Please help

Thanks....

 

[xhoster]

Hi,

I'm trying to write a PERL program that parses a binary file which
essentially contains the following format:

<float><int><int><int><int><int>

This "record" is repeated throughout the binary file. I'm using
unpack() to extract that information, however, the results I'm seeing
are getting corrupted in PERL v5.8.0 (Linux). I ran the same PERL code
on v5.6.1 Linux (I downloaded it from ActiveState, but didn't "install"
it on my linux system...just cd'ed to the location where the perl
executable lives from this package) and everything works fine. Has
this been observed by anyone else? Is this a known bug in v5.8.0?

I don't know, but I'd try calling binmode on the filehandle, or doing
"use bytes;" at the top of the script.

Here's the code snippet that I wrote:

$template = "f I5";
$recordsize = length(pack($template,( )));

Is the size reported the same for both versions?

open(IN, "binFile.bin") or die "Cannot open file";
while (read(IN, $record, $recordsize))
{
($f, $i_1, $i_2, $i_3, $i_4, $i_5) = unpack($template, $record);
print "f=$f, i_1=$i_1, i_2=$i_2, i_3=$i_3, i_4=$i_4, i_5=$i_5\n";
}

If anyone can help I will greatly appreciate it. Thanks.

Exactly what corruption are you seeing under 5.8? What is the correct
data you see under 5.6?

Xho

 

[Avalon1178]

I don't know, but I'd try calling binmode on the filehandle, or doing
"use bytes;" at the top of the script.

I'll give this a shot...

Is the size reported the same for both versions?

Yes, record size is 24 bytes, as expected, on both systems.

Exactly what corruption are you seeing under 5.8? What is the correct
data you see under 5.6?

I don't have the data files in front me, but essentially, on the first
loop, the data looks good, a float, followed by 5 ints. Then the next
few iterations shows very large numbers that doesn't make sense, then
somewhere in between, I see normal values again, etc.... Basically,
evidence that the data coming in is not being placed correctly in the
variables, like something was offset...

What doesn't make sense though is that I ran the same script using v5.6
on Linux and it works. I ran the same script under cygwin perl v5.8.7
I believe and THAT also worked....

Is there something unusual that I'm doing in the script? Or is this a
known problem?

 

[Avalon1178]

Hey Xho,

That works! Thanks!

AV

 

[xhoster]

Hey Xho,

That works! Thanks!

You are welcome, but please do quote some context.

For the record, which worked? The "use bytes;" or the
binmode?

Thanks,

Xho

 

[Avalon1178]

It was the binmode operation, but I ended up using both just in case.

I was actually a bit surprised, since the documentation says that the
binmode operation in Linux is essentially a no-op. However, that is
not the case for me....

AV

 

[xhoster]

It was the binmode operation, but I ended up using both just in case.

I was actually a bit surprised, since the documentation says that the
binmode operation in Linux is essentially a no-op. However, that is
not the case for me....

from perldoc perluniintro:

If your locale environment variables ("LANGUAGE", "LC_ALL",
"LC_CTYPE", "LANG") contain the strings 'UTF-8' or 'UTF8',
regardless of case, then the default encoding of your STDIN, STDOUT,
and STDERR and of any sub- sequent file open, is UTF-8.

I've gotten screwed by this more times than I care to think about. Usually
not in Perl, but rather with the system tools such as "sort" or "join". It
seems like every few months they change the way these variables take
precedence over each other, or some system wide setting changes which
overrules whatever I had to do last time to make it work. It's gotten to
the point that I'm tempted to have my .*rc files loop on every stinking
environment variable with a capital L in it, and unset them all.

Xho