Unpack - getting values directly into the correct variables

Discussion in 'Perl Misc' started by PugetSoundSylvia@gmail.com, Apr 10, 2008.

  1. Guest

    Hello all,

    I'm just starting to get into perl, so please forgive me if I'm asking
    something obvious.

    I'm using the unpack function to parse through a fixed length file.
    It's working well, but there's lots and lots of fields, and I'd like
    to make the code more readable.

    What I have now is this:

    $LOGFILE = "text.log";
    open(LOGFILE) or die("Could not open log file.");

    # Define the record format for unpack function
    $BaseBSPFileTemplate =
    "A3" # RecordType Field 0
    ."A8" # SequenceNumber Field 1
    ."A2" # RecordTypeSuffix Field 2
    ."A6" # CreateDate Field 3
    ."A6" # TransactionNumber Field 4
    ."A15" # DataNumber Field 5
    ."A98" # UpdateDate Field 6
    ;

    while (<LOGFILE>) {

    @fields = unpack( $BaseBSPFileTemplate, $_ );

    $RecordType = $fields[0];
    $SequenceNumber = $fields[1];
    $RecordTypeSuffix = $fields[2];

    ... and so forth ...

    Is there a better way to do this - one where the unpack function
    itself would automatically split it into the actual variables
    ($RecordType, $SequenceNumber, $RecordTypeSuffix, etc) - instead of me
    having to have the section that has a bunch of rows like this:

    $RecordType = $fields[0];

    Thanks much for any advice!!

    Sylvia
    , Apr 10, 2008
    #1
    1. Advertising

  2. Guest

    wrote:
    >
    > while (<LOGFILE>) {
    >
    > @fields = unpack( $BaseBSPFileTemplate, $_ );
    >
    > $RecordType = $fields[0];
    > $SequenceNumber = $fields[1];
    > $RecordTypeSuffix = $fields[2];
    >
    > ... and so forth ...
    >
    > Is there a better way to do this - one where the unpack function
    > itself would automatically split it into the actual variables
    > ($RecordType, $SequenceNumber, $RecordTypeSuffix, etc) - instead of me
    > having to have the section that has a bunch of rows like this:


    You can assign directly to a list of variables:

    my ( $RecordType,
    $SequenceNumber,
    $RecordTypeSuffix,
    # ....
    ) = unpack( $BaseBSPFileTemplate, $_ );


    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    The costs of publication of this article were defrayed in part by the
    payment of page charges. This article must therefore be hereby marked
    advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
    this fact.
    , Apr 10, 2008
    #2
    1. Advertising

  3. wrote:
    >
    > I'm just starting to get into perl, so please forgive me if I'm asking
    > something obvious.
    >
    > I'm using the unpack function to parse through a fixed length file.
    > It's working well, but there's lots and lots of fields, and I'd like
    > to make the code more readable.


    I see that Xho has answered your unpack question but ...

    > What I have now is this:


    You should really have these two lines at the beginning of your program:

    use warnings;
    use strict;

    > $LOGFILE = "text.log";


    Which means that you have to declare this variable:

    my $LOGFILE = 'text.log';

    > open(LOGFILE) or die("Could not open log file.");


    And you should really be using the three argument form of open(). As
    well, you should include the $! variable in the error message so you
    know why it failed:

    open LOGFILE, '<', $LOGFILE or die "Could not open '$LOGFILE' $!";



    John
    --
    Perl isn't a toolbox, but a small machine shop where you
    can special-order certain sorts of tools at low cost and
    in short order. -- Larry Wall
    John W. Krahn, Apr 10, 2008
    #3
  4. wrote in
    news:a9c1aa0a-7f6f-4d7d-94e9-dc474b1ba039
    @t12g2000prg.googlegroups.co
    m:

    > Hello all,
    >
    > I'm just starting to get into perl, so please forgive me if I'm
    > asking something obvious.
    >
    > I'm using the unpack function to parse through a fixed length
    > file. It's working well, but there's lots and lots of fields, and
    > I'd like to make the code more readable.
    >
    > What I have now is this:
    >
    > $LOGFILE = "text.log";
    > open(LOGFILE) or die("Could not open log file.");
    >
    > # Define the record format for unpack function
    > $BaseBSPFileTemplate =
    > "A3" # RecordType Field 0
    > ."A8" # SequenceNumber Field 1
    > ."A2" # RecordTypeSuffix Field 2
    > ."A6" # CreateDate Field 3
    > ."A6" # TransactionNumber Field 4
    > ."A15" # DataNumber Field 5
    > ."A98" # UpdateDate Field 6
    > ;
    >
    > while (<LOGFILE>) {
    >
    > @fields = unpack( $BaseBSPFileTemplate, $_ );
    >
    > $RecordType = $fields[0];
    > $SequenceNumber = $fields[1];
    > $RecordTypeSuffix = $fields[2];


    #!/usr/bin/perl

    use strict;
    use warnings;

    my @fields = qw( RecordType SequenceNumber RecordTypeSuffix
    CreateDate TransactionNumber DataNumber UpdateDate );

    my %formats;
    @formats{ @fields } = qw( A3 A8 A2 A6 A6 A15 A98 );

    my $unpack_tmpl = join( '', @formats{ @fields } );

    while( <DATA> ) {
    last if /^\s+$/;
    my %obs;
    @obs{ @fields } = unpack $unpack_tmpl;

    print "$_\t$obs{$_}\n" for @fields;

    }

    __END__
    00011111111223333334444445555555555555556666666666666666666666666666
    66666666666666666666666666666666666666666666666666666666666666666666
    66


    --
    A. Sinan Unur <>
    (remove .invalid and reverse each component for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://www.rehabitation.com/clpmisc/
    A. Sinan Unur, Apr 10, 2008
    #4
  5. A. Sinan Unur wrote:
    > wrote in
    > news:a9c1aa0a-7f6f-4d7d-94e9-dc474b1ba039
    > @t12g2000prg.googlegroups.co
    > m:
    >
    >> Hello all,
    >>
    >> I'm just starting to get into perl, so please forgive me if I'm
    >> asking something obvious.
    >>
    >> I'm using the unpack function to parse through a fixed length
    >> file. It's working well, but there's lots and lots of fields, and
    >> I'd like to make the code more readable.
    >>
    >> What I have now is this:
    >>
    >> $LOGFILE = "text.log";
    >> open(LOGFILE) or die("Could not open log file.");
    >>
    >> # Define the record format for unpack function
    >> $BaseBSPFileTemplate =
    >> "A3" # RecordType Field 0
    >> ."A8" # SequenceNumber Field 1
    >> ."A2" # RecordTypeSuffix Field 2
    >> ."A6" # CreateDate Field 3
    >> ."A6" # TransactionNumber Field 4
    >> ."A15" # DataNumber Field 5
    >> ."A98" # UpdateDate Field 6
    >> ;
    >>
    >> while (<LOGFILE>) {
    >>
    >> @fields = unpack( $BaseBSPFileTemplate, $_ );
    >>
    >> $RecordType = $fields[0];
    >> $SequenceNumber = $fields[1];
    >> $RecordTypeSuffix = $fields[2];

    >
    > #!/usr/bin/perl
    >
    > use strict;
    > use warnings;
    >
    > my @fields = qw( RecordType SequenceNumber RecordTypeSuffix
    > CreateDate TransactionNumber DataNumber UpdateDate );
    >
    > my %formats;
    > @formats{ @fields } = qw( A3 A8 A2 A6 A6 A15 A98 );
    >
    > my $unpack_tmpl = join( '', @formats{ @fields } );
    >
    > while( <DATA> ) {
    > last if /^\s+$/;
    > my %obs;
    > @obs{ @fields } = unpack $unpack_tmpl;


    Don't forget the variable that you want to unpack:

    @obs{ @fields } = unpack $unpack_tmpl, $_;


    > print "$_\t$obs{$_}\n" for @fields;
    >
    > }
    >
    > __END__
    > 00011111111223333334444445555555555555556666666666666666666666666666
    > 66666666666666666666666666666666666666666666666666666666666666666666
    > 66



    John
    --
    Perl isn't a toolbox, but a small machine shop where you
    can special-order certain sorts of tools at low cost and
    in short order. -- Larry Wall
    John W. Krahn, Apr 10, 2008
    #5
  6. "John W. Krahn" <> wrote in news:khtLj.27964
    $pb5.25660@edtnps89:

    > A. Sinan Unur wrote:

    ....

    >> @obs{ @fields } = unpack $unpack_tmpl;

    >
    > Don't forget the variable that you want to unpack:
    >
    > @obs{ @fields } = unpack $unpack_tmpl, $_;


    I didn't ;-) From perldoc -f unpack:

    unpack TEMPLATE,EXPR
    unpack TEMPLATE
    ....
    If EXPR is omitted, unpacks the $_ string.

    Sinan

    --
    A. Sinan Unur <>
    (remove .invalid and reverse each component for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://www.rehabitation.com/clpmisc/
    A. Sinan Unur, Apr 11, 2008
    #6
  7. [A complimentary Cc of this posting was sent to

    <>], who wrote in article <>:
    > $BaseBSPFileTemplate =
    > "A3" # RecordType Field 0
    > ."A8" # SequenceNumber Field 1
    > ."A2" # RecordTypeSuffix Field 2
    > ."A6" # CreateDate Field 3
    > ."A6" # TransactionNumber Field 4
    > ."A15" # DataNumber Field 5
    > ."A98" # UpdateDate Field 6
    > ;
    >
    > while (<LOGFILE>) {
    >
    > @fields = unpack( $BaseBSPFileTemplate, $_ );
    >
    > $RecordType = $fields[0];
    > $SequenceNumber = $fields[1];
    > $RecordTypeSuffix = $fields[2];


    What a horror... Here is an example (Audio::FindChunks):

    my $wav_header = <<EOH;
    a4 # header: 'RIFF'
    V # size: Size of what follows
    a4 # type: 'WAVE'

    a4 # type1: 'fmt ' subchunk
    V # size1: Size of the rest of subchunk
    v # format: 1 for pcm
    v # channels: 2 stereo 1 mono
    V # frequency
    V # bytes_per_sec
    v # bytes_per_sample
    v # bits_per_sample_channel

    a4 # type2: 'data' subchunk
    V # sizedata: Size of the rest of subchunk
    EOH

    my @wav_fields = ($wav_header =~ /^\s*\w+\s*#\s*(\w+)/mg);

    $wav_header =~ s/#.*//g; # For v5.005
    ....
    @vals{@wav_fields} = unpack $wav_header, $in or ...

    Hope this helps,
    Ilya
    Ilya Zakharevich, Apr 11, 2008
    #7
  8. A. Sinan Unur wrote:
    > "John W. Krahn" <> wrote in news:khtLj.27964
    > $pb5.25660@edtnps89:
    >
    >> A. Sinan Unur wrote:

    > ...
    >
    >>> @obs{ @fields } = unpack $unpack_tmpl;

    >> Don't forget the variable that you want to unpack:
    >>
    >> @obs{ @fields } = unpack $unpack_tmpl, $_;

    >
    > I didn't ;-) From perldoc -f unpack:
    >
    > unpack TEMPLATE,EXPR
    > unpack TEMPLATE
    > ...
    > If EXPR is omitted, unpacks the $_ string.


    That must be new for 5.10 cause it doesn't work in 5.8.8

    $ perl -le'
    $_ = q[1234567890123456789012345678901234567890];
    print for unpack q[a4 a7 a3];
    '
    Not enough arguments for unpack at -e line 3, near "q[a4 a7 a3];"
    Execution of -e aborted due to compilation errors.



    John
    --
    Perl isn't a toolbox, but a small machine shop where you
    can special-order certain sorts of tools at low cost and
    in short order. -- Larry Wall
    John W. Krahn, Apr 11, 2008
    #8
  9. On 2008-04-10 23:50, Ilya Zakharevich <> wrote:
    ><>], who wrote in article <>:
    >> $BaseBSPFileTemplate =
    >> "A3" # RecordType Field 0
    >> ."A8" # SequenceNumber Field 1
    >> ."A2" # RecordTypeSuffix Field 2
    >> ."A6" # CreateDate Field 3
    >> ."A6" # TransactionNumber Field 4
    >> ."A15" # DataNumber Field 5
    >> ."A98" # UpdateDate Field 6
    >> ;
    >>
    >> while (<LOGFILE>) {
    >>
    >> @fields = unpack( $BaseBSPFileTemplate, $_ );
    >>
    >> $RecordType = $fields[0];
    >> $SequenceNumber = $fields[1];
    >> $RecordTypeSuffix = $fields[2];

    >
    > What a horror...


    You could have phrased that more politely :).

    Actually, I think that's pretty low on the horror-scale. Tedious and
    redundant, yes, but I've seen much worse code.

    > Here is an example (Audio::FindChunks):
    >
    > my $wav_header = <<EOH;
    > a4 # header: 'RIFF'
    > V # size: Size of what follows
    > a4 # type: 'WAVE'
    >
    > a4 # type1: 'fmt ' subchunk
    > V # size1: Size of the rest of subchunk
    > v # format: 1 for pcm
    > v # channels: 2 stereo 1 mono
    > V # frequency
    > V # bytes_per_sec
    > v # bytes_per_sample
    > v # bits_per_sample_channel
    >
    > a4 # type2: 'data' subchunk
    > V # sizedata: Size of the rest of subchunk
    > EOH
    >
    > my @wav_fields = ($wav_header =~ /^\s*\w+\s*#\s*(\w+)/mg);
    >
    > $wav_header =~ s/#.*//g; # For v5.005
    > ...
    > @vals{@wav_fields} = unpack $wav_header, $in or ...


    Clever. Maybe a bit too clever for production code. At least I would add
    a comment about the format (so that the next maintainer doesn't break
    it) and why it works.

    I'd use an array of arrays instead:

    #!/usr/bin/perl
    use warnings;
    use strict;


    my @BaseBSPFileFormat = (
    ["RecordType", "A3" ],
    ["SequenceNumber", "A8" ],
    ["RecordTypeSuffix", "A2" ],
    ["CreateDate", "A6" ],
    ["TransactionNumber", "A6" ],
    ["DataNumber", "A15" ],
    ["UpdateDate", "A98" ],
    );
    my $BaseBSPFileTemplate = join('', map $_->[1], @BaseBSPFileFormat);
    my @BaseBSPFileFields = map $_->[0], @BaseBSPFileFormat;

    while (<DATA>) {

    my %fields;
    @fields{@BaseBSPFileFields} = unpack( $BaseBSPFileTemplate, $_ );

    for (@BaseBSPFileFields) {
    print "$_: $fields{$_}\n";
    }
    print "\n"
    }
    __DATA__
    111222222223344444455555566666666666666677777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777
    111222222223344444455555566666666666666677777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777

    hp
    Peter J. Holzer, Apr 12, 2008
    #9
  10. On Apr 11, 5:31 am, "John W. Krahn" <> wrote:
    > A. Sinan Unur wrote:
    > > "John W. Krahn" <> wrote in news:khtLj.27964
    > > unpack TEMPLATE,EXPR
    > > unpack TEMPLATE
    > > ...
    > > If EXPR is omitted, unpacks the $_ string.

    >
    > That must be new for 5.10


    And about $EXPLETIVE time! It's always bugged me that unpack didn't do
    this.
    Brian McCauley, Apr 12, 2008
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Elezar Simeon Papo

    ValueError: need more than 3 values to unpack

    Elezar Simeon Papo, Jan 21, 2006, in forum: Python
    Replies:
    3
    Views:
    1,411
    Elezar Simeon Papo
    Feb 7, 2006
  2. Replies:
    2
    Views:
    27,291
  3. fscked
    Replies:
    8
    Views:
    347
    Bruno Desthuilliers
    Apr 11, 2007
  4. Replies:
    9
    Views:
    942
  5. Chris Rebert
    Replies:
    1
    Views:
    678
    Bobby
    May 28, 2009
Loading...

Share This Page