dealing with modified csv data format

Discussion in 'Perl Misc' started by nun, Mar 20, 2007.

  1. nun

    nun Guest

    I wrote a script which parses csv data files which were provided to me
    in the following format.

    WL,WA6B,SCRAPE,N,5.99,6.85,8.56,4.98
    WL,W30,RULE6"",N,7.90,47.90,6.05,3.15

    Note the comma field separator and that some fields have double quotes
    in the data.

    My script read this data in as follows and all was well:

    ###############################
    # reading data in from file
    my (@AoA);
    while ( <> ) {
    chomp;
    push @AoA, [ split /,/ ];
    }
    ################################

    Now suddenly I'm being given the data in a slightly different format,
    still comma separated but now all fields are enclosed by double quotes,
    even those which have double quotes in the data, such as:

    "WL","WA6B","SCRAPE","N","5.99","6.85","8.56","4.98"
    "WL","W30","RULE6""","N","7.90","47.90","6.05","3.15"


    Is there an easy way I can modify my code to deal with this new format,
    or must I start over with something like TEXT::CSV instead of split? I'm
    no perl guru and am hoping for an easy cure :)

    DB
    nun, Mar 20, 2007
    #1
    1. Advertising

  2. nun

    Paul Lalli Guest

    On Mar 20, 12:06 pm, nun <> wrote:
    > I wrote a script which parses csv data files which were provided to me
    > in the following format.
    >
    > WL,WA6B,SCRAPE,N,5.99,6.85,8.56,4.98
    > WL,W30,RULE6"",N,7.90,47.90,6.05,3.15
    >
    > Note the comma field separator and that some fields have double quotes
    > in the data.
    >
    > My script read this data in as follows and all was well:
    >
    > ###############################
    > # reading data in from file
    > my (@AoA);
    > while ( <> ) {
    > chomp;
    > push @AoA, [ split /,/ ];
    > }
    > ################################
    >
    > Now suddenly I'm being given the data in a slightly different format,
    > still comma separated but now all fields are enclosed by double quotes,
    > even those which have double quotes in the data, such as:
    >
    > "WL","WA6B","SCRAPE","N","5.99","6.85","8.56","4.98"
    > "WL","W30","RULE6""","N","7.90","47.90","6.05","3.15"
    >
    > Is there an easy way I can modify my code to deal with this new format,
    > or must I start over with something like TEXT::CSV instead of split? I'm
    > no perl guru and am hoping for an easy cure :)


    Why not just keep it the way it is, but then remove all the beginning/
    ending quotes from each field after you've split?

    push @AoA, [ split /,/ ];
    s/^"// and s/"$// for @{$AoA[-1]};

    Paul Lalli
    Paul Lalli, Mar 20, 2007
    #2
    1. Advertising

  3. nun

    nun Guest

    Paul Lalli wrote:
    > On Mar 20, 12:06 pm, nun <> wrote:
    >> I wrote a script which parses csv data files which were provided to me
    >> in the following format.
    >>
    >> WL,WA6B,SCRAPE,N,5.99,6.85,8.56,4.98
    >> WL,W30,RULE6"",N,7.90,47.90,6.05,3.15
    >>
    >> Note the comma field separator and that some fields have double quotes
    >> in the data.
    >>
    >> My script read this data in as follows and all was well:
    >>
    >> ###############################
    >> # reading data in from file
    >> my (@AoA);
    >> while ( <> ) {
    >> chomp;
    >> push @AoA, [ split /,/ ];
    >> }
    >> ################################
    >>
    >> Now suddenly I'm being given the data in a slightly different format,
    >> still comma separated but now all fields are enclosed by double quotes,
    >> even those which have double quotes in the data, such as:
    >>
    >> "WL","WA6B","SCRAPE","N","5.99","6.85","8.56","4.98"
    >> "WL","W30","RULE6""","N","7.90","47.90","6.05","3.15"
    >>
    >> Is there an easy way I can modify my code to deal with this new format,
    >> or must I start over with something like TEXT::CSV instead of split? I'm
    >> no perl guru and am hoping for an easy cure :)

    >
    > Why not just keep it the way it is, but then remove all the beginning/
    > ending quotes from each field after you've split?
    >
    > push @AoA, [ split /,/ ];
    > s/^"// and s/"$// for @{$AoA[-1]};
    >
    > Paul Lalli
    >
    >


    That works, and fits the bill as easy - thanks!

    DB
    nun, Mar 20, 2007
    #3
  4. nun wrote:
    > I wrote a script which parses csv data files which were provided to me
    > in the following format.
    >
    > WL,WA6B,SCRAPE,N,5.99,6.85,8.56,4.98
    > WL,W30,RULE6"",N,7.90,47.90,6.05,3.15
    >
    > Note the comma field separator and that some fields have double quotes
    > in the data.
    >
    > My script read this data in as follows and all was well:
    >
    > ###############################
    > # reading data in from file
    > my (@AoA);
    > while ( <> ) {
    > chomp;
    > push @AoA, [ split /,/ ];
    > }
    > ################################
    >
    > Now suddenly I'm being given the data in a slightly different format,
    > still comma separated but now all fields are enclosed by double quotes,
    > even those which have double quotes in the data, such as:
    >
    > "WL","WA6B","SCRAPE","N","5.99","6.85","8.56","4.98"
    > "WL","W30","RULE6""","N","7.90","47.90","6.05","3.15"
    >
    >
    > Is there an easy way I can modify my code to deal with this new format,
    > or must I start over with something like TEXT::CSV instead of split? I'm
    > no perl guru and am hoping for an easy cure :)


    $ perl -le'
    $_ = q[WL,WA6B,SCRAPE,N,5.99,6.85,8.56,4.98];
    my @x = split /,/;
    print "@x";
    $_ = q["WL","WA6B","SCRAPE","N","5.99","6.85","8.56","4.98"];
    my @y = map /\A"(.+)"\z/ ? $1 : $_, split /,/;
    print "@y";
    '
    WL WA6B SCRAPE N 5.99 6.85 8.56 4.98
    WL WA6B SCRAPE N 5.99 6.85 8.56 4.98




    John
    --
    Perl isn't a toolbox, but a small machine shop where you can special-order
    certain sorts of tools at low cost and in short order. -- Larry Wall
    John W. Krahn, Mar 20, 2007
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. brzak
    Replies:
    5
    Views:
    1,521
    Boogie
    Dec 1, 2008
  2. Tim Chase
    Replies:
    0
    Views:
    112
    Tim Chase
    Sep 4, 2013
  3. MRAB
    Replies:
    0
    Views:
    112
  4. Tim Chase
    Replies:
    0
    Views:
    129
    Tim Chase
    Sep 4, 2013
  5. Tim Chase
    Replies:
    0
    Views:
    108
    Tim Chase
    Sep 4, 2013
Loading...

Share This Page