reading binary files

Discussion in 'Perl Misc' started by Zoran, Dec 6, 2003.

  1. Zoran

    Zoran Guest

    Hello,
    I have problems with unpacking binary data(win98,activestate perl).
    The procedure is very simple:
    1.Packing ascii data as binary data.
    2.Unpacking binary data.
    3.Printing the output.

    This is my input file:
    0.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
    0.2 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
    1.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
    1.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
    2.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
    2.3 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
    2.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
    3.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
    3.2 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0
    3.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0

    This is the output:
    0.1 0 0 0 0 0 0 0 0 0 0 0 0 0
    0.2 0 0 0 0 0 0 0 0 0 0 0 0 0
    1.1 0 0 0 0 0 0 0 0 0 0 0 0 0
    1.5 0 0 0 0 0 0 0 0 0 0 0 0 0
    2.1 0 0 0 0 0 0 0 0 0 0 0 0 0
    2.3 0 0 0 0 0 0 0 0 0 0 0 0 0
    2.5 0 0 0 0 0 0 0 0 0 0 0 0 0
    3.1 0 0 0 0 0 0 0 0 0 0 0 0 0
    3.2 0 0 0 0 0 0 0 0 0 0 0 0 0
    3.5 0 0 0 0 0 0 0 0 0 0 0 0 0

    As you can see only the first numbers are correct.Maybe I am making
    mistakes ,when I try to pack the data in a special format
    @var= pack ("d I d11",$buf); .


    This is the code:
    ______________________________________________________
    print "\n ******************************\n";

    open (B ,"< stats.dat") or die "failed opening stats.dat \n";
    open (C ,"> bin.dat") or die "failed creating bin.dat \n";

    while (my $buf = <B>)
    {
    chomp $buf;
    @var2 = pack("d I d11",$buf);
    print C "@var2\n";
    }
    close(C);
    close(B);

    open (B ,"< bin.dat") or die "failed opening bin.dat \n";
    open (C ,"> asc.dat") or die "failed opening asc.dat \n";

    binmode(B);

    while ($buf2=<B>)
    #while (read B,$buf2,102)
    {
    chomp $buf2;
    @var = unpack("d I d11",$buf2);
    print C "@var\n";
    print " @var\n";
    }
    close(C);
    close(B);

    print " Operation done\n";
    print " ******************************\n";
    __________________________________________________
     
    Zoran, Dec 6, 2003
    #1
    1. Advertisements

  2. Zoran

    Greg Bacon Guest

    : I have problems with unpacking binary data(win98,activestate perl).
    : The procedure is very simple:
    : 1.Packing ascii data as binary data.
    : 2.Unpacking binary data.
    : 3.Printing the output.
    : [...]

    See below.

    #! perl

    use warnings;
    use strict;

    my @input = (
    "0.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
    "0.2 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
    "1.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
    "1.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
    "2.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
    "2.3 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
    "2.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
    "3.1 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
    "3.2 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
    "3.5 11 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0",
    );

    my $bin;
    open $bin, ">", "bin.dat" or die "$0: open >bin.dat: $!\n";
    binmode $bin;

    my $line = 0;
    for (@input) {
    ++$line;

    my @data = split;

    my($tag,$n) = splice @data, 0, 2;
    unless (@data == $n) {
    my $got = @data;
    warn "$0: line $line: n = $n but $got field(s)\n";
    next;
    }

    my $packed = pack "d I d$n", $tag, $n, @data;
    print $bin length($packed), "!" => $packed;
    }

    close $bin or warn "$0: close bin.dat: $!\n";

    sub check {
    my $expect = shift;
    my @data = split ' ', shift;

    return unless @data == @$expect;

    for (@$expect) {
    my $datum = shift @data;

    return unless $_ == $datum;
    }

    1;
    }

    sub getrec {
    my $fh = shift;

    my $bytes;

    my $ch;
    my $status = read $fh, $ch, 1;
    if ($status == 0) {
    return 0;
    }
    elsif (not defined $status) {
    warn "$0: read: $!";
    return;
    }

    while ($ch ne "!") {
    last if $ch eq "!";

    if ($ch =~ /\D/) {
    warn "$0: unexpected '$ch'";
    return;
    }

    $bytes .= $ch;

    $status = read $fh, $ch, 1;
    if ($status == 0) {
    warn "$0: premature end-of-file";
    return;
    }
    elsif (not defined $status) {
    warn "$0: read: $!";
    return;
    }
    }

    unless (defined $bytes) {
    warn "$0: number of bytes undefined";
    return;
    }

    $status = read $fh, my($chunk), $bytes;
    if ($status == 0) {
    warn "$0: premature end-of-file";
    return;
    }
    elsif (not defined $status) {
    warn "$0: read: $!";
    return;
    }
    elsif ($bytes != length $chunk) {
    warn "$0: incomplete read";
    return;
    }

    $chunk;
    }

    open $bin, "<", "bin.dat" or die "$0: open bin.dat: $!\n";
    binmode $bin;

    while (defined ($_ = getrec $bin)) {
    last unless $_;

    my @data = unpack "d I d*", $_;

    my $input = shift @input;

    if (check \@data, $input) {
    print "[@data]:\n",
    " MATCH\n";
    }
    else {
    print "[$input]:\n",
    " ERROR\n";

    s/(.)/sprintf "%02x", ord $1/ges;
    print " [$_]:\n",
    " [@data]\n";
    }

    push @input, $input;
    }

    __END__

    Hope this helps,
    Greg
     
    Greg Bacon, Dec 6, 2003
    #2
    1. Advertisements

  3. You should include the $! variable in the error message so you know why
    open() failed. Since you are running Windows you should use binmode()
    on the C filehandle.

    pack() takes a list and converts it to a string while unpack() takes a
    string and converts it to a list. You are passing a list of one element
    to pack. You need to separate the individual numbers into a list for it
    to work correctly.

    while ( <B> ) {
    my $var2 = pack 'd I d11', split;
    print C "$var2\n";
    }



    John
     
    John W. Krahn, Dec 6, 2003
    #3
  4. Note that relying on a newline to delimit binary data makes no sense as
    one of your 'd' or 'I' formatted numbers may contain the same bit
    pattern as the "\n" character.



    John
     
    John W. Krahn, Dec 6, 2003
    #4
  5. Zoran

    Ben Morrow Guest

    Don't retype code. Even error messages.
    perldoc perldiag

    Or add 'use diagnostics;' to the top of your script to have perl look
    them up for you.

    Ben
     
    Ben Morrow, Dec 6, 2003
    #5
  6. Zoran

    Zoran Guest

    Thank you very much!
    I think I have managed it with your help.

    Look at this:

    @var2 = pack("d I d11",split(" ",$buf));

    Finally my output file matches my input file,though the code
    is not very elegant.
    If I am using "use strict" the script won't even start and crushes
    with warnings like this:

    Possible unintended interpolation of @var2 in string at one.pl line17.
    Possible unintended interpolation of @var in string at one.pl line 32.
    Possible unintended interpolation of @var in string at one.pl line 33.
    Global symbol "@var2" requires explicit package name at one.pl line16.
    Global symbol "@var2" requires explicit package name at one.pl line17.
    Global symbol "$buf2" requires explicit package name at one.pl line27.
    Global symbol "$buf2" requires explicit package name at one.pl line30.
    Global symbol "@var" requires explicit package name at one.pl line 31.
    Global symbol "$buf2" requires explicit package name at one.pl line31.
    Global symbol "@var" requires explicit package name at one.pl line 32.
    Global symbol "@var" requires explicit package name at one.pl line 33.
    Execution of one.pl aborted due to compilation errors.

    So where can I find the explanation of these error warnings?
     
    Zoran, Dec 6, 2003
    #6
  7. Zoran

    Greg Bacon Guest

    : sub getrec {
    : my $fh = shift;
    :
    : my $bytes;
    :
    : my $ch;
    : my $status = read $fh, $ch, 1;
    : if ($status == 0) {
    : return 0;
    : }
    : elsif (not defined $status) {
    : warn "$0: read: $!";
    : return;
    : }
    :
    : while ($ch ne "!") {
    : last if $ch eq "!";
    :
    : if ($ch =~ /\D/) {
    : warn "$0: unexpected '$ch'";
    : return;
    : }
    :
    : $bytes .= $ch;
    :
    : $status = read $fh, $ch, 1;
    : if ($status == 0) {
    : warn "$0: premature end-of-file";
    : return;
    : }
    : elsif (not defined $status) {
    : warn "$0: read: $!";
    : return;
    : }
    : }
    :
    : unless (defined $bytes) {
    : warn "$0: number of bytes undefined";
    : return;
    : }
    :
    : $status = read $fh, my($chunk), $bytes;
    : if ($status == 0) {
    : warn "$0: premature end-of-file";
    : return;
    : }
    : elsif (not defined $status) {
    : warn "$0: read: $!";
    : return;
    : }
    : elsif ($bytes != length $chunk) {
    : warn "$0: incomplete read";
    : return;
    : }
    :
    : $chunk;
    : }

    Yuck, so much repetition. There's gotta be a better way.

    Greg
     
    Greg Bacon, Dec 6, 2003
    #7
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.