Processing multiple delimiter files

Discussion in 'Perl Misc' started by ollie, Mar 13, 2006.

  1. ollie

    ollie Guest

    Can someone help with a perl construct for this. I am trying to perform
    a SQL type update on a delimiter file. I want to use the value in one
    file to update matching row in the other file. Kind of what you'll do
    in a relational database system, but I need to do this outside of the
    database.

    Example
    1st file
    1,32, ,houston
    3,40, ,albany
    3,44, ,nyc

    2nd file
    1,Texas
    3,NY

    Result:
    1,32,Texas,houston
    3,40,NY,albany
    3,44,NY,albany
     
    ollie, Mar 13, 2006
    #1
    1. Advertising

  2. ollie wrote:
    > Can someone help with a perl construct for this.


    Help? Sure, but first show us what you have tried, and how it fails to
    satisfy your needs. Please comply with the posting guidelines for this
    group when posting your code.

    > I am trying to perform
    > a SQL type update on a delimiter file. I want to use the value in one
    > file to update matching row in the other file. Kind of what you'll do
    > in a relational database system, but I need to do this outside of the
    > database.
    >
    > Example
    > 1st file
    > 1,32, ,houston
    > 3,40, ,albany
    > 3,44, ,nyc
    >
    > 2nd file
    > 1,Texas
    > 3,NY
    >
    > Result:
    > 1,32,Texas,houston
    > 3,40,NY,albany
    > 3,44,NY,albany


    How did 'nyc' change to 'albany' at the 3:rd line?

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
     
    Gunnar Hjalmarsson, Mar 13, 2006
    #2
    1. Advertising

  3. ollie

    ollie Guest

    Sorry, the output should be

    1,32,Texas,houston
    3,40,NY,albany
    3,44,NY,nyc
     
    ollie, Mar 13, 2006
    #3
  4. ollie

    Matt Garrish Guest

    "ollie" <> wrote in message
    news:...
    > Sorry, the output should be
    >
    > 1,32,Texas,houston
    > 3,40,NY,albany
    > 3,44,NY,nyc
    >


    Please quote context when replying.

    Where is your code? How does DBD-CSV or Tie-Handle-CSV not help do what you
    want, for example?

    Matt
     
    Matt Garrish, Mar 13, 2006
    #4
  5. ollie

    Eric Bohlman Guest

    "ollie" <> wrote in news:1142286451.557668.38820
    @i40g2000cwc.googlegroups.com:

    > Can someone help with a perl construct for this. I am trying to perform
    > a SQL type update on a delimiter file. I want to use the value in one
    > file to update matching row in the other file. Kind of what you'll do
    > in a relational database system, but I need to do this outside of the
    > database.
    >
    > Example
    > 1st file
    > 1,32, ,houston
    > 3,40, ,albany
    > 3,44, ,nyc
    >
    > 2nd file
    > 1,Texas
    > 3,NY
    >
    > Result:
    > 1,32,Texas,houston
    > 3,40,NY,albany
    > 3,44,NY,albany


    Why not use DBD::CSV and actually do it in SQL? If for some reason you
    don't want to use SQL, Data::Table and some of its relatives provide join
    operations.
     
    Eric Bohlman, Mar 13, 2006
    #5
  6. ollie

    ollie Guest

    I would like to use hashmap data structure. I can create the hash as
    follows

    open(<IN>, "> 2ndfile") || die;
    @rows = <IN>;
    close(IN);

    %data_row = ();
    foreach (@rows) (
    $data_row{$rows[0] = $rows[1];
    }

    This is were I get stuck
     
    ollie, Mar 14, 2006
    #6
  7. "ollie" <> wrote in news:1142295081.669913.102070
    @p10g2000cwp.googlegroups.com:

    [ You really ought to read the posting guidelines and start actually
    responding to the questions people ask you in their quest to help
    you. You should also quote some context when you reply.
    See <URL:http://cfaj.freeshell.org/google/>
    ]

    > I would like to use hashmap data structure. I can create the hash as
    > follows
    >


    use strict;
    use warnings;

    missing.

    > open(<IN>, "> 2ndfile") || die;


    You have just overwritten '2ndfile'.

    > @rows = <IN>;


    It does not make sense to try to read from a filehandle which you have
    opened for writing only.

    Don't rewrite code ... copy and paste it into your post.

    The posting guidelines explain how to post code.

    > close(IN);
    >
    > %data_row = ();
    > foreach (@rows) (
    > $data_row{$rows[0] = $rows[1];
    > }
    >
    > This is were I get stuck


    In your case, I suspect that the best solution to your problem involves
    using DBD::CSV. The following crufty "solution" should illustrate my
    lack of skills:

    #!/usr/bin/perl

    use strict;
    use warnings;

    my %states = (
    1 => 'Texas',
    3 => 'NY'
    );

    my %cities = (
    houston => { state_code => 1, city_code => 32 },
    albany => { state_code => 3, city_code => 40 },
    nyc => { state_code => 3, city_code => 44 },
    );

    for my $k ( keys %cities ) {
    $cities{$k}->{state_name} = $states{ $cities{$k}->{state_code} };
    {
    local $" = ', ';
    my %record = %{ $cities{$k} };
    print "@record{qw(state_code city_code state_name)}, $k\n";
    }
    }

    __END__

    D:\Home\asu1\UseNet\clpmisc> h
    3, 40, NY, albany
    3, 44, NY, nyc
    1, 32, Texas, houston


    Sinan

    --
    A. Sinan Unur <>
    (remove .invalid and reverse each component for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
     
    A. Sinan Unur, Mar 14, 2006
    #7
  8. ollie <> wrote:

    > I can create the hash as
    > follows
    >
    > open(<IN>, "> 2ndfile") || die;



    No you can't.

    It does not even compile.

    All of the angle bracket characters should be removed from that line.

    Why did you put them there?


    > @rows = <IN>;
    > close(IN);
    >
    > %data_row = ();
    > foreach (@rows) (



    There is another syntax error.

    You need a curly brace, not a parenthesis.

    Is this your real code?


    > $data_row{$rows[0] = $rows[1];



    There is yet another syntax error, missing a closing curly somewhere.


    > }
    >
    > This is were I get stuck



    You got stuck way before here (where you claim that you can
    create the hash).

    Here is one way to create the hash:

    -----------------------
    #!/usr/bin/perl
    use warnings;
    use strict;

    open(IN, '2ndfile') or die "could not open '2ndfile' $!";

    my($key, %data_row);
    while ( <IN> ) {
    chomp;
    if ( $. % 2 ) # an odd-numbered line
    { $key = $_ }
    else # an even-numbered line
    { $data_row{$key} = $_ }
    }
    close(IN);

    print "$_ => $data_row{$_}\n" for sort keys %data_row;
    -----------------------


    But I'd do it like this:

    -----------------------
    #!/usr/bin/perl
    use warnings;
    use strict;

    open(IN, '2ndfile') or die "could not open '2ndfile' $!";

    my %data_row = map {chomp; $_} <IN>;
    close(IN);

    print "$_ => $data_row{$_}\n" for sort keys %data_row;
    -----------------------


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Mar 14, 2006
    #8
  9. ollie

    ollie Guest

    Thanks Tad for the suggestions. The sample code was for illustration
    and not a real code. I'll will look into using DBD::CSV. I though there
    might be a way to do achieve the objective using hashmaps all the way

    Thanks again.

    Tad McClellan wrote:
    > ollie <> wrote:
    >
    > > I can create the hash as
    > > follows
    > >
    > > open(<IN>, "> 2ndfile") || die;

    >
    >
    > No you can't.
    >
    > It does not even compile.
    >
    > All of the angle bracket characters should be removed from that line.
    >
    > Why did you put them there?
    >
    >
    > > @rows = <IN>;
    > > close(IN);
    > >
    > > %data_row = ();
    > > foreach (@rows) (

    >
    >
    > There is another syntax error.
    >
    > You need a curly brace, not a parenthesis.
    >
    > Is this your real code?
    >
    >
    > > $data_row{$rows[0] = $rows[1];

    >
    >
    > There is yet another syntax error, missing a closing curly somewhere.
    >
    >
    > > }
    > >
    > > This is were I get stuck

    >
    >
    > You got stuck way before here (where you claim that you can
    > create the hash).
    >
    > Here is one way to create the hash:
    >
    > -----------------------
    > #!/usr/bin/perl
    > use warnings;
    > use strict;
    >
    > open(IN, '2ndfile') or die "could not open '2ndfile' $!";
    >
    > my($key, %data_row);
    > while ( <IN> ) {
    > chomp;
    > if ( $. % 2 ) # an odd-numbered line
    > { $key = $_ }
    > else # an even-numbered line
    > { $data_row{$key} = $_ }
    > }
    > close(IN);
    >
    > print "$_ => $data_row{$_}\n" for sort keys %data_row;
    > -----------------------
    >
    >
    > But I'd do it like this:
    >
    > -----------------------
    > #!/usr/bin/perl
    > use warnings;
    > use strict;
    >
    > open(IN, '2ndfile') or die "could not open '2ndfile' $!";
    >
    > my %data_row = map {chomp; $_} <IN>;
    > close(IN);
    >
    > print "$_ => $data_row{$_}\n" for sort keys %data_row;
    > -----------------------
    >
    >
    > --
    > Tad McClellan SGML consulting
    > Perl programming
    > Fort Worth, Texas
     
    ollie, Mar 14, 2006
    #9
  10. ollie

    Brad Baxter Guest

    ollie wrote:
    > Thanks Tad for the suggestions. The sample code was for illustration
    > and not a real code. I'll will look into using DBD::CSV. I though there
    > might be a way to do achieve the objective using hashmaps all the way
    >


    This sample code is for illustration.

    Contents of ./qt:

    #!/usr/local/bin/perl
    use warnings;
    use strict;
    use Text::parseWords;

    my @f1 = `cat file1`;
    my @f2 = `cat file2`;
    die unless @f1 and @f2;

    my %h;
    for( @f2 ) {
    chomp;
    my @a = parse_line( ',', '', $_ );
    $h{ $a[0] } = $a[1];
    }

    for( @f1 ) {
    my @a = parse_line( ',', '', $_ );
    $a[2] = $h{ $a[0] };
    print join ',' => @a;
    }

    >> cat file1

    1,32, ,houston
    3,40, ,albany
    3,44, ,nyc

    >> cat file2

    1,Texas
    3,NY

    >> ./qt

    1,32,Texas,houston
    3,40,NY,albany
    3,44,NY,nyc


    Regards,

    --
    Brad
     
    Brad Baxter, Mar 14, 2006
    #10
  11. ollie <> wrote:

    > Thanks Tad for the suggestions.



    You are welcome.

    I will accept you having a look at the Posting Guidelines that are
    posted here frequently as an easy way for you to pay me back. :)


    > The sample code was for illustration
    > and not a real code.



    It is disrespectful to post code that is not real.

    Lots of people will waste their time fixing syntax errors that
    do not even really exist!

    Please post only real code (copy/paste it, do not try to re-type it).



    [ snip TOFU ]

    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Mar 14, 2006
    #11
  12. ollie

    DJ Stunks Guest

    Brad Baxter wrote:
    >
    > This sample code is for illustration.
    >
    > Contents of ./qt:
    >
    > #!/usr/local/bin/perl
    > use warnings;
    > use strict;
    > use Text::parseWords;
    >
    > my @f1 = `cat file1`;
    > my @f2 = `cat file2`;


    -- OUCH! --^^^^^^^^^^^

    let's not illustrate shelling out if we can avoid it, huh?

    -jp
     
    DJ Stunks, Mar 14, 2006
    #12
  13. ollie

    ollie Guest

    If you paid attention to my first posting, the help i seeked for was on
    the perl construct. I would have said so if i needed help with the
    code.

    Sorry, that your focused on the code more than the problem.
     
    ollie, Mar 14, 2006
    #13
  14. "ollie" <> wrote in news:1142363940.036412.259190
    @j33g2000cwa.googlegroups.com:

    [ The fact that you are still not quoting any context as you have been
    asked reflects poorly on you ]

    > If you paid attention to my first posting, the help i seeked
    > for was on the perl construct. I would have said so if i needed
    > help with the code.
    >
    > Sorry, that your focused on the code more than the problem.


    Sorry that you insist on being uncooperative.

    You are expected to post short, self-contained code that does makes it
    easy for others to help you.

    Bye!

    Sinan

    --
    A. Sinan Unur <>
    (remove .invalid and reverse each component for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://mail.augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
     
    A. Sinan Unur, Mar 14, 2006
    #14
  15. ollie

    Brad Baxter Guest

    DJ Stunks wrote:
    > Brad Baxter wrote:
    > >
    > > This sample code is for illustration.
    > >
    > > Contents of ./qt:
    > >
    > > #!/usr/local/bin/perl
    > > use warnings;
    > > use strict;
    > > use Text::parseWords;
    > >
    > > my @f1 = `cat file1`;
    > > my @f2 = `cat file2`;

    >
    > -- OUCH! --^^^^^^^^^^^
    >
    > let's not illustrate shelling out if we can avoid it, huh?
    >


    Given that Perl is a glue language, and that in some
    circumstances shelling out to an external utility is a
    good solution, and that the above is an illustration of
    how one might do that in Perl (though arguably an
    incomplete one), I am therefore unapologetic. Sorry.

    perl -F','
    -lane'if($|){$F[2]=$_{$F[0]};print"@F"}else{$_{$F[0]}=$F[1]}eof&&++$|;$"=","'
    file2 file1

    There. No shelling out.

    Cheers,

    --
    Brad
     
    Brad Baxter, Mar 14, 2006
    #15
  16. ollie

    Keith Keller Guest

    On 2006-03-14, ollie <> wrote:
    > If you paid attention to my first posting, the help i seeked for was on
    > the perl construct.


    Your first posting did not contain any Perl constructs.

    > I would have said so if i needed help with the code.


    Apparently not, since the code that you did post had syntax errors.

    > Sorry, that your focused on the code more than the problem.


    It sounds like the code is part of the problem; the other part is that
    you are still not following the posting guidelines to which you have
    been repeatedly referred. If you want free help, you should probably
    exercise some common courtesy when posting.

    --keith

    --
    -francisco.ca.us
    (try just my userid to email me)
    AOLSFAQ=http://wombat.san-francisco.ca.us/cgi-bin/fom
    see X- headers for PGP signature information
     
    Keith Keller, Mar 14, 2006
    #16
  17. ollie

    DJ Stunks Guest

    Brad Baxter wrote:
    > I am therefore unapologetic. Sorry.


    I don't understand -- you're sorry you're not apologetic? Or, like,
    unapologetic but sorry nonetheless?

    Peter: Chris, everything I say is a lie. Except that. And that. And
    that. And that. And that. And that. And that. And that.

    -jp
     
    DJ Stunks, Mar 14, 2006
    #17
  18. ollie <> wrote:
    > If you paid



    If who?


    > attention to my first posting,



    It seems pretty clear to me that I was paying attention
    to your first posting.

    Rest assured that that will not happen again.


    > the help i seeked for was on
    > the perl construct.



    What you posted was not a Perl (nor perl) construct.

    The perl parser is the definition of Perl, and since it was
    not accepted by perl, it was not (yet) Perl.


    > I would have said so if i needed help with the
    > code.



    It sure did look to me like you needed help with the code,
    because the code could not be compiled or executed.


    > Sorry, that your focused on the code more than the problem.



    Sorry you have volunteered to become invisible.

    So long!


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Mar 14, 2006
    #18
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    4
    Views:
    999
    M.E.Farmer
    Feb 13, 2005
  2. Bruno Desthuilliers
    Replies:
    8
    Views:
    340
    Hendrik van Rooyen
    Jul 26, 2007
  3. Christopher Steele

    processing input from multiple files

    Christopher Steele, Oct 14, 2010, in forum: Python
    Replies:
    2
    Views:
    281
    John Posner
    Oct 14, 2010
  4. Replies:
    3
    Views:
    105
    Brian McCauley
    Apr 26, 2007
  5. Replies:
    20
    Views:
    315
    News123
    Jan 2, 2009
Loading...

Share This Page