reading binary files

[Zoran]

Hello,
I have problems with unpacking binary data(win98,activestate perl).
The procedure is very simple:
1.Packing ascii data as binary data.
2.Unpacking binary data.
3.Printing the output.

This is my input file:
0.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
0.2 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
1.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
1.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
2.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
2.3 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
2.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
3.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
3.2 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
3.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

This is the output:
0.1 0 0 0 0 0 0 0 0 0 0 0 0 0
0.2 0 0 0 0 0 0 0 0 0 0 0 0 0
1.1 0 0 0 0 0 0 0 0 0 0 0 0 0
1.5 0 0 0 0 0 0 0 0 0 0 0 0 0
2.1 0 0 0 0 0 0 0 0 0 0 0 0 0
2.3 0 0 0 0 0 0 0 0 0 0 0 0 0
2.5 0 0 0 0 0 0 0 0 0 0 0 0 0
3.1 0 0 0 0 0 0 0 0 0 0 0 0 0
3.2 0 0 0 0 0 0 0 0 0 0 0 0 0
3.5 0 0 0 0 0 0 0 0 0 0 0 0 0

As you can see only the first numbers are correct.Maybe I am making
mistakes ,when I try to pack the data in a special format
@var= pack ("d I d11",$buf); .


This is the code:
______________________________________________________
print "\n ******************************\n";

open (B ,"< stats.dat") or die "failed opening stats.dat \n";
open (C ,"> bin.dat") or die "failed creating bin.dat \n";

while (my $buf = <B>)
{
chomp $buf;
@var2 = pack("d I d11",$buf);
print C "@var2\n";
}
close(C);
close(B);

open (B ,"< bin.dat") or die "failed opening bin.dat \n";
open (C ,"> asc.dat") or die "failed opening asc.dat \n";

binmode(B);

while ($buf2=<B>)
#while (read B,$buf2,102)
{
chomp $buf2;
@var = unpack("d I d11",$buf2);
print C "@var\n";
print " @var\n";
}
close(C);
close(B);

print " Operation done\n";
print " ******************************\n";
__________________________________________________

 

[Greg Bacon]

: I have problems with unpacking binary data(win98,activestate perl).
: The procedure is very simple:
: 1.Packing ascii data as binary data.
: 2.Unpacking binary data.
: 3.Printing the output.
: [...]

See below.

#! perl

use warnings;
use strict;

my @input = (
"0.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
"0.2 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
"1.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
"1.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
"2.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
"2.3 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
"2.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
"3.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
"3.2 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
"3.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
);

my $bin;
open $bin, ">", "bin.dat" or die "$0: open >bin.dat: $!\n";
binmode $bin;

my $line = 0;
for (@input) {
++$line;

my @data = split;

my($tag,$n) = splice @data, 0, 2;
unless (@data == $n) {
my $got = @data;
warn "$0: line $line: n = $n but $got field(s)\n";
next;
}

my $packed = pack "d I d$n", $tag, $n, @data;
print $bin length($packed), "!" => $packed;
}

close $bin or warn "$0: close bin.dat: $!\n";

sub check {
my $expect = shift;
my @data = split ' ', shift;

return unless @data == @$expect;

for (@$expect) {
my $datum = shift @data;

return unless $_ == $datum;
}

1;
}

sub getrec {
my $fh = shift;

my $bytes;

my $ch;
my $status = read $fh, $ch, 1;
if ($status == 0) {
return 0;
}
elsif (not defined $status) {
warn "$0: read: $!";
return;
}

while ($ch ne "!") {
last if $ch eq "!";

if ($ch =~ /\D/) {
warn "$0: unexpected '$ch'";
return;
}

$bytes .= $ch;

$status = read $fh, $ch, 1;
if ($status == 0) {
warn "$0: premature end-of-file";
return;
}
elsif (not defined $status) {
warn "$0: read: $!";
return;
}
}

unless (defined $bytes) {
warn "$0: number of bytes undefined";
return;
}

$status = read $fh, my($chunk), $bytes;
if ($status == 0) {
warn "$0: premature end-of-file";
return;
}
elsif (not defined $status) {
warn "$0: read: $!";
return;
}
elsif ($bytes != length $chunk) {
warn "$0: incomplete read";
return;
}

$chunk;
}

open $bin, "<", "bin.dat" or die "$0: open bin.dat: $!\n";
binmode $bin;

while (defined ($_ = getrec $bin)) {
last unless $_;

my @data = unpack "d I d*", $_;

my $input = shift @input;

if (check \@data, $input) {
print "[@data]:\n",
" MATCH\n";
}
else {
print "[$input]:\n",
" ERROR\n";

s/(.)/sprintf "%02x", ord $1/ges;
print " [$_]:\n",
" [@data]\n";
}

push @input, $input;
}

__END__

Hope this helps,
Greg

 

[John W. Krahn]

I have problems with unpacking binary data(win98,activestate perl).
The procedure is very simple:
1.Packing ascii data as binary data.
2.Unpacking binary data.
3.Printing the output.

This is my input file:
0.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

[snip]

This is the output:
0.1 0 0 0 0 0 0 0 0 0 0 0 0 0

[snip]

As you can see only the first numbers are correct.Maybe I am making
mistakes ,when I try to pack the data in a special format
@var= pack ("d I d11",$buf); .

This is the code:
______________________________________________________
print "\n ******************************\n";

open (B ,"< stats.dat") or die "failed opening stats.dat \n";
open (C ,"> bin.dat") or die "failed creating bin.dat \n";

You should include the $! variable in the error message so you know why
open() failed. Since you are running Windows you should use binmode()
on the C filehandle.

while (my $buf = <B>)
{
chomp $buf;
@var2 = pack("d I d11",$buf);
print C "@var2\n";
}

pack() takes a list and converts it to a string while unpack() takes a
string and converts it to a list. You are passing a list of one element
to pack. You need to separate the individual numbers into a list for it
to work correctly.

while ( <B> ) {
my $var2 = pack 'd I d11', split;
print C "$var2\n";
}

close(C);
close(B);

John

 

[John W. Krahn]

pack() takes a list and converts it to a string while unpack() takes a
string and converts it to a list. You are passing a list of one element
to pack. You need to separate the individual numbers into a list for it
to work correctly.

while ( <B> ) {
my $var2 = pack 'd I d11', split;
print C "$var2\n";
}

Note that relying on a newline to delimit binary data makes no sense as
one of your 'd' or 'I' formatted numbers may contain the same bit
pattern as the "\n" character.



John

 

[Ben Morrow]

If I am using "use strict" the script won't even start and crushes
with warnings like this:

Possible unintended interpolation of @var2 in string at one.pl line17.

Don't retype code. Even error messages.

Possible unintended interpolation of @var in string at one.pl line 32.
Possible unintended interpolation of @var in string at one.pl line 33.
Global symbol "@var2" requires explicit package name at one.pl line16.

So where can I find the explanation of these error warnings?

perldoc perldiag

Or add 'use diagnostics;' to the top of your script to have perl look
them up for you.

Ben

 

[Zoran]

Note that relying on a newline to delimit binary data makes no sense as
one of your 'd' or 'I' formatted numbers may contain the same bit
pattern as the "\n" character.



John

Thank you very much!
I think I have managed it with your help.

Look at this:

@var2 = pack("d I d11",split(" ",$buf));

Finally my output file matches my input file,though the code
is not very elegant.
If I am using "use strict" the script won't even start and crushes
with warnings like this:

Possible unintended interpolation of @var2 in string at one.pl line17.
Possible unintended interpolation of @var in string at one.pl line 32.
Possible unintended interpolation of @var in string at one.pl line 33.
Global symbol "@var2" requires explicit package name at one.pl line16.
Global symbol "@var2" requires explicit package name at one.pl line17.
Global symbol "$buf2" requires explicit package name at one.pl line27.
Global symbol "$buf2" requires explicit package name at one.pl line30.
Global symbol "@var" requires explicit package name at one.pl line 31.
Global symbol "$buf2" requires explicit package name at one.pl line31.
Global symbol "@var" requires explicit package name at one.pl line 32.
Global symbol "@var" requires explicit package name at one.pl line 33.
Execution of one.pl aborted due to compilation errors.

So where can I find the explanation of these error warnings?

 

[Greg Bacon]

: sub getrec {
: my $fh = shift;
:
: my $bytes;
:
: my $ch;
: my $status = read $fh, $ch, 1;
: if ($status == 0) {
: return 0;
: }
: elsif (not defined $status) {
: warn "$0: read: $!";
: return;
: }
:
: while ($ch ne "!") {
: last if $ch eq "!";
:
: if ($ch =~ /\D/) {
: warn "$0: unexpected '$ch'";
: return;
: }
:
: $bytes .= $ch;
:
: $status = read $fh, $ch, 1;
: if ($status == 0) {
: warn "$0: premature end-of-file";
: return;
: }
: elsif (not defined $status) {
: warn "$0: read: $!";
: return;
: }
: }
:
: unless (defined $bytes) {
: warn "$0: number of bytes undefined";
: return;
: }
:
: $status = read $fh, my($chunk), $bytes;
: if ($status == 0) {
: warn "$0: premature end-of-file";
: return;
: }
: elsif (not defined $status) {
: warn "$0: read: $!";
: return;
: }
: elsif ($bytes != length $chunk) {
: warn "$0: incomplete read";
: return;
: }
:
: $chunk;
: }

Yuck, so much repetition. There's gotta be a better way.

Greg